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Introduction 


Neuroengineering is a rapidly evolving field at the intersection of 
neuroscience, engineering, and technology, aiming to develop innovative 
solutions to understand, interface with, and treat the nervous system. This 


' 


book, "Introduction to Neuroengineering," is designed as a comprehensive 
resource to provide students, educators, and professionals with a thorough 
understanding of the foundational concepts, current technologies, and future 
directions in this exciting domain. By leveraging the principles of 
neuroengineering, we can push the boundaries of medical technology, enhance 
quality of life, and gain deeper insights into the workings of the human brain 
and nervous system. 

The book is structured into several chapters, each dealing with key topics 
within neuroengineering. We start with the basics, such as neuron morphology 
and functionality, action potentials, and the nervous system. These 
foundational concepts set the stage for more advanced discussions on the 
Hodgkin-Huxley model and mathematical modeling of neurons. 
Understanding these principles is crucial as they form the bedrock upon which 
neuroengineering techniques are built. 


Chapter Breakdown 
Neuron Morphology and Functionality 


This chapter introduces the structure and function of neurons, the fundamental 


units of the nervous system. It covers the different types of neurons, their roles, 
and how they communicate through electrical and chemical signals. 


Hodgkin-Huxley Model and Mathematical Modeling of 
Neurons 

Here, we delve into the Hodgkin-Huxley model, a seminal framework for 
understanding the electrical characteristics of neurons. This chapter also 
explores various mathematical models used to simulate neuronal behavior. 


Neural Network Basics 


This chapter covers the basics of neural networks, including coefficient 
feedback and supervised and unsupervised learning. These concepts are 
critical for developing algorithms that mimic or interface with the human 
brain. 


EEG and MEG Hardware and Software 


Electroencephalography (EEG) and Magnetoencephalography (MEG) are 
non-invasive techniques for measuring brain activity. This chapter discusses 
the hardware and software used in EEG and MEG, noise considerations, and 
the advantages and disadvantages of these methods. It also explains the 10-20 
system for electrode placement and how EEG data is classified into different 
bands and event-related potentials (ERPs). 


JMRI and fNIRS 


Functional Magnetic Resonance Imaging (fMRI) and functional Near-Infrared 
Spectroscopy (fNIRS) are imaging techniques used to measure brain activity. 
This chapter covers what these techniques measure, the hardware and software 
involved, and their advantages and disadvantages. 


Brain-Computer Interfaces (BCTs) 


BCIs are systems that enable direct communication between the brain and 
external devices. This chapter explores the main characteristics of BCIs, major 
applications, the current state of the art, and the pros and cons of various BCI 
technologies. 


Neurostimulation and Neuromodulation 


This chapter discusses techniques such as Transcranial Magnetic Stimulation 
(TMS) and Deep Brain Stimulation (DBS), including their hardware, software, 
advantages, disadvantages, and current technological advancements. 


Visual Neuroengineering 


This chapter focuses on the eye, visual nerve, and components such as cones, 
rods, retina, and cornea. It also covers visual-neuro diseases and their 
technological interventions. 


Psychophysics and Virtual Reality 


Psychophysics explores the relationship between physical stimuli and sensory 
perception. This chapter also examines the use of Virtual Reality (VR) as a 
tool for psychophysics and the various types of tests used in this field. 


Eye Tracking Technology 


Eye trackers measure eye positions and movements. This chapter discusses 
how eye trackers work, their effectiveness, hardware considerations, main 
vendors, and their advantages and disadvantages. 


Auditory Neuroengineering 


This final chapter covers the ear and auditory system, including nerve 
distributions, cochlear prostheses, and related hardware and software. 


Neurological Disorders and Technological Solutions 


Here, we provide a description of various neurological disorders, physiological 
explanations of how and why they occur, contributing factors, and current 
technological applications to solve these problems or improve the quality of 
life for affected individuals. 


Learning Activities and Lab Examples 


In addition, each chapter includes a section dedicated to learning activities that 
can be implemented in classroom settings. These activities are based on the 
principles of collaboration, representation, safe space, and autonomy. Students 
can deepen their understanding of neuroengineering concepts through 
practical, hands-on experience by engaging in these activities. 

Additionally, we propose two lab examples per chapter that apply the 
principles learned in each section. These lab examples involve hardware and 
software designed to be attainable for students at the appropriate level. These 


labs reinforce theoretical knowledge and provide practical skills in using 
neuroengineering tools and techniques. 

To further enhance the learning experience, we have accompanied this book 
with a GitHub repository containing self-directed lab examples. These labs are 
highly based on the principles of autonomy, allowing students to discover their 
own path of learning and apply basic principles to their interests. 

In summary, "Introduction to Neuroengineering" is more than just a textbook; 
it is a gateway to understanding and innovating within one of modern science 
and technology's most dynamic and impactful fields. We hope this resource 
will inspire, educate, and empower you to contribute to the exciting and ever- 
evolving field of neuroengineering. The journey through this book is only the 
beginning, and we encourage you to continue exploring, experimenting, and 
innovating. 


Disclaimer 


e All graphics used in this work have been utilized with the owners' 
explicit permission or in accordance with the appropriate usage 
licenses for images. 

e All graphic images presented at the beginning of each chapter were 
generated using ChatGPT 4.0 with the following prompt: "Create a 
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Chapter 1 


Decoding Neurons: Structure, 
Function and Connectivity 


Introduction and Learning Objectives 


Hello reader, in this first chapter, we will explore the fundamental building 
block of the brain: neurons. This chapter will provide a foundational 
understanding of neurons by examining their structure and function. Then, the 
chapter will discuss the mechanisms behind how neurons communicate and 
the intricacies involved in neural processing. The chapter ends by discussing 
the organization of the central and peripheral nervous systems, highlighting 
the functional divisions. This foundational knowledge is crucial for anyone 
aiming to innovate in the field of neural engineering, as it provides the 
background needed to develop new technologies and interventions. By the end 
of this chapter, you will be able to: 


Describe neuron morphology, explain how each component con- 
tributes to neural signaling, and identify different types of neurons. 


Explain the process of action potential generation and propagation. 
Understand the organization of the central and peripheral nervous 
system. 


Introduction to Neurons 
Definition and Morphology of Neurons 


Our entire body is composed of cells, and in the brain, these cells are called 
neurons. The neurons are the fundamental units of the brain and nervous 
system and are responsible for transmitting information throughout the entire 
body. These signals control everything in our body, from basic reflexes to 
complex thoughts, emotions, and behaviors. The neurons’ anatomical 
structure, or morphology, is important because it is designed to transmit these 
signals as fast and efficiently as possible and is dictated by the specific function 
that neurons are supposed to carry out. The main morphological features of a 
neuron are the dendrites, the soma or cell body, the axon, the myelin sheath, 
and the axon terminals (Fig. 1.1 [1], [2]). 
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The dendrites are the first stop for neurons to transmit signals through the body. 
The dendrites are branching processes that extend from the soma of the 
neurons. These branching-like structures are sometimes referred to as dendritic 
trees or dendritic arbors due to their structures resembling those of tree 
branches. The complexity of the branching of the dendrites can vary depending 
on the type of neuron. The function of the dendrites is to receive signals from 
other neurons and transmit those signals to the soma. The dendrites are crucial 


axon terminal (from 
different neuron) 


dendrites 


nucleus 
cell body 


Figure 1.1: Neuron Morphology (left) with close up of axon terminals (top right) and segment 
of dendrite (bottom left) [1],[2] 
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for integrating signal inputs to determine the neuron output. The dendrites can 
be thought of as an antenna on a satellite dish. Like how antennae receive 
signals from different sources, the dendrite receives many incoming signals 
from various other neurons and relays those signals to the soma. 

Once the signal travels down the dendrites, it will reach the soma. The soma, 
the neuron's cell body, contains the organelles typically found in cells, such as 
the nucleus, the endoplasmic reticulum, and mitochondria. These organelles 
maintain the health of the cell and integrate all the signals received from the 
dendrites to determine the neuron’s output. The soma can be considered a 
company's central operation office. Many different players, in this case, 
organelles, are working together to control activities and make decisions to 
direct actions. Once the soma can integrate the signals received from the 
dendrites, the soma then sends the output signal to the axon. 

The axon is a long projection whose length can vary from a short length of less 
than a millimeter to a meter or more. The axon originates from a specialized 
part of the soma called the axon hillock. This specialized part of the soma is 
crucial because it is where the neuron initiates action potentials, which we will 
discuss later in this chapter. The axon transmits the signals away from the soma 
to other neurons in the body, whether in the brain, muscles, or glands. The axon 
is essential for communicating the signals across various distances in the body. 
This part of the neuron can be thought of as a delivery truck that must travel 
to deliver packages. In this case, the package is an electrical signal, and the 
recipient is various body parts. The axon has multiple other structures to ensure 
its signal can be delivered as fast as possible. These structures include the 
myelin sheath and the nodes of Ranvier. 

The myelin sheath is a fatty insulating layer formed by glial cells. 
Oligodendrocytes are the glial cells that myelinate the axons of neurons in the 
central nervous system (CNS). In contrast, Schwann cells are the glial cells 
that myelinate the axons of neurons in the peripheral nervous system (PNS). 
Diseases such as multiple sclerosis, which will be discussed more in Chapter 
12, can lead to the destruction of these glial cells, resulting in demyelination 
of the axon and the axon's inability to transmit electrical signals to other 
neurons. The function of the myelin sheath is to increase the speed of the signal 
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as it travels down the axon. The axon can speed up this transmission using the 
nodes of Ranvier, which are gaps in the myelin sheath where the axonal 
membrane is exposed. When the signal travels down the axon, it “jumps” from 
one node of Ranvier to the next. This “jumping” allows for the signal to 
transmit efficiently and quickly. The process of the signal moving down the 
axon in this way is called saltatory conduction. The myelin sheath can be 
thought of as the insulation of electrical wires. Just as the wire insulation 
speeds up electrical transmission in a circuit, the myelin sheath insulates the 
axon to speed up neural signal transmission. Ranvier's nodes can be considered 
stepping stones in a stream that allows you to jump from stone to stone across 
a stream. The nodes will enable the signal to “jump” from one node to the next 
along the axon. Once the signal jumps down the nodes of Ranvier, it reaches 
its destination at the axon terminals, also known as the terminal boutons or 
synaptic boutons. This part of the neuron transmits the signal to the next 
neuron. The axon terminals contain additional structures, such as synaptic 
vehicles and mitochondria, which support signal transmission. This will be 
discussed in more detail later in this chapter. 

While all neurons typically have the same morphology, their organization can 
look very different. These differences end up producing various types of 
neurons. 


Types of Neurons 


Neurons can be classified based on their structure and function. When looking 
at how the anatomical structures of neurons are organized, they can be 
classified as unipolar, bipolar, or multipolar, as depicted in Fig. 1.2 [3]. 
Unipolar neurons exhibit one single process extending from the soma. Bipolar 
neurons have two processes, typically one axon and one dendrite, that extend 
in opposite directions from the cell body. Finally, multipolar neurons, the most 
common type, have one axon extending from one end of the soma and several 
dendrites extending from the other. 
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information to the central Figure 1.2: Types of Neurons [3] 

nervous system (CNS). For 

example, dorsal root ganglion neurons are unipolar sensory neurons that take 
sensory inputs such as pain, temperature, and proprioception from peripheral 
tissues to the CNS. In these neurons, a process comes from the soma that 
bifurcates (splits), with one branch extending to the sensory receptors in the 
skin or other tissues and the other branch extending to the spinal cord. When 
a unipolar neuron bifurcates, such as those for the DRG, they are called pseudo 
unipolar neurons. The structure of these neurons is essential for them to 
efficiently transmit sensory information over long distances. Not all sensory 
neurons are unipolar; some can be bipolar. However, sensory neurons are not 
usually multipolar. 

The nervous system also employs motor neurons and interneurons. Motor 
neurons, or efferent neurons, take signals from the CNS to target cells in 
muscles and glands to carry out actions. Motor neurons are typically multipolar 
neurons with multiple processes extending to other neurons and target cells. 
The multipolar structure of these neurons is crucial for allowing them to 
receive inputs from various sources and integrate them to control muscles 
efficiently. The nervous system uses interneurons to link sensory and motor 
neurons to communicate signals from the CNS to the peripheral nervous 
system (PNS). Interneurons are also typically multipolar, allowing them to 
receive sensory information, integrate it, and transmit an output signal to motor 
neurons. Understanding the morphology of neurons and their structural 
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diversity is foundational knowledge for understanding the dynamic process of 
neuronal communication. 


The Action Potential 


Membrane Potential 


The membrane potential in a neuron is the electrical potential difference across 
the cell membrane. This potential is created due to an unequal distribution of 
charged ions between the cell's intracellular (inside) and extracellular (outside) 
environments. The membrane potential is crucial in the neuron’s ability to 
generate and propagate electrical signals, called action potentials. The 
membrane has a resting potential of -70 millivolts (mV). This negative 
potential is established and maintained by ion concentration gradients, 
membrane permeability, and the sodium-potassium pump. 


Concentration Gradients 


A concentration gradient is established when a solute's concentration varies 
across different regions in a solution. When discussing neurons, our solutes are 
ions, the solution is the body’s fluids, and the regions are the intracellular and 
extracellular fluids. The cell’s phospholipid bilayer acts as the membrane 
separating these regions. The concentration gradient in the neuron is due to a 
difference in ion concentrations and the membrane's selective permeability. 
The ions that are primarily involved in the creation of this gradient are sodium 
(Na*), potassium (K*), chloride (Cl), and calcium (Ca”*). Na* and Ca** ions 
have higher concentrations outside the cell than inside, whereas K* and Cl 
have higher concentrations inside the cell. 


Permeability 

The cell membrane of the neuron is selectively permeable. This means the 
membrane regulates the passage of certain substances across it, which is 
essential for maintaining homeostasis. The membrane's permeability is 
established due to its structure. The cell membrane is a phospholipid bilayer 
where two layers of phospholipids containing a hydrophilic head and a 
hydrophobic tail come together. The hydrophobic tails make up the interior of 
the bilayer and act as a barrier for water-soluble substances, while the 
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hydrophilic heads make up the exterior of the bilayer and interact with the 
extracellular and intracellular environments. The membrane is composed of 
proteins, cholesterol, and carbohydrate chains. Integral proteins, especially ion 
channels embedded in the bilayer, facilitate selective permeability through 
passive and active transport mechanisms. This will be discussed in more detail 
later in the chapter. 


Sodium Potassium Pump 


The sodium-potassium ATPase pump is a membrane protein that uses energy 
from adenosine triphosphate (ATP) to transport ions against their 
concentration gradient from an area of low concentration to an area of high 
concentration. Specifically, this pump moves Na’* ions out of the cell and K* 
ions into the cell, transporting them against their concentration gradients. The 
transport of these ions is essential for maintaining their concentration gradient 
inside and outside the cell. The pump operates through a cycle that involves 
binding and moving ions across the membrane. First, the pump binds three Na* 
from inside the cell and then hydrolyzes ATP to adenosine diphosphate (ADP). 
This hydrolysis breaks one of the phosphate bonds on the ATP, releasing 
energy in phosphorylation. Phosphorylation adds a phosphate group to the 
pump, causing a conformational change in the pump allowing it to transport 
Na’* ions into the extracellular space. The pump binds two K* ions from the 
extracellular fluid in this new conformation. It uses dephosphorylation to 
undergo another conformation change, returning the pump to its original shape 
and exposing the K* to the intracellular fluid, where it can be released. Since 
the pump uses energy to move ions from one side of the membrane to the other, 
it is called active transport. 

Now that you have established the foundation of the membrane potential, we 
can discuss how ion channels orchestrate the flow of these ions across the 
membrane to drive dynamic changes in the electrical charge necessary for 
neurons to communicate with one another. 


Ton Channels 


The proteins embedded in the membrane make up ion channels where ions can 
enter or leave the cell. There are different channels, such as voltage-gated ion 
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channels, ligand-gated channels, mechanically-gated channels, and leak 
channels. Voltage-gated channels open in response to changes in the membrane 
potential. Ligand-gated channels open in response to a molecule (ligand) 
binding. Mechanically-gated channels respond to mechanical stimuli such as 
stretching or pressure changes. Leak channels are always open, allowing ions 
to move down their concentration gradients across the membrane. 
Voltage-gated ion channels are crucial for neural communication and undergo 
a conformation change in response to electrical stimuli, allowing them to open 
to allow ions to pass through. These channels have specific characteristics that 
make them selective to ions, which only allows specific ions to pass through. 
For example, there is a voltage-gated ion channel that is selective to Na* and 
selective to K*. The ion channel pore size can determine the selectivity of these 
channels. The K* channel is narrower than the Na* channel and wouldn’t be 
big enough for the larger Na* ion to pass through. Additionally, the channels 
are lined with specific amino acids that create a charge that will favor the 
passage of particular ions. For example, the Na* channels have a negatively 
charged amino acid near the entrance to attract the positively charged Na‘ ions. 
The channel selectivity ensures that the appropriate ions can pass through at 
the correct times for the neurons to communicate by generating action 
potentials. The movement of the ions down these channels without energy is a 
type of transport called passive transport. In this case, this type of passive 
transport is called facilitated diffusion. The ions can travel down the channels 
because they are moving down their concentration gradient from an area of 
high concentration to an area of lower concentration. Other examples of 
channels that use passive transport are ligand-gated channels and leak 
channels. In these channels, the ions' concentration gradient determines the 
rate at which passive transport occurs and does not require energy [4]. 


Action Potential Phases 


The way that neurons can communicate with each other is by generating action 
potentials. The action potential is an electrical event in neurons involving rapid 
changes in potential across the neuron’s cell membrane. The action potential 
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has three distinct sequential phases: depolarization, repolarization, and the 
refractory period. These phases can be seen in Fig. 1.3 [5]. 


Depolarization 


The neurons start at a resting potential of —70 mV due to unequal ion 
distributions inside and outside the cell. The neuron membrane contains 
ligand-gated Na* channels that open once the neuron receives an excitatory 
electrical impulse, called excitatory postsynaptic potential (ESPS), from 
another neuron [6]. Once an excitatory neurotransmitter binds to these gate 
channels, they open, allowing Na‘ to influx (enter) the cell. As Na‘ continues 
to enter the cell, the membrane potential becomes more positive. An action 
potential will occur once enough Na+ enters the cell to reach a threshold of — 
55 mV. 

The action potential is an all-or-nothing event. If the neuron does not receive 
sufficient ESPS to reach the threshold, an action potential will not occur, and 
the membrane potential will return to resting potential. These failed action 
potentials are referred to as “Failed Initiations.” Once the threshold is met, 
voltage-gated Na* channels open, allowing more Na’ to influx into the cell to 
reach a membrane potential of around 40 mV. At this point in the action 
potential, the Na* channels begin to close, and voltage-gated K* opens. 


Repolarization 

The closure of the Na* channels ensures that no more Na‘ enters the cell, and 
the opening of the K* channels allows K* ions to efflux out of the cell. The 
efflux of K* leads to the membrane potential getting more negative to try and 
restore the membrane potential to its resting value. The K* channels can be 
opened for a prolonged time, causing excessive amounts of K* to leave the 
cell. This causes the membrane potential to become more negative than the 
resting potential, called hyperpolarization. The voltage-gated potassium 
channels eventually close and restore resting membrane potential. 
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Refractory Period 

The inactivation of voltage-gated sodium channels and the hyperpolarization 
of voltage-gated potassium channels results in a refractory period. This feature 
of the action potential may seem counterintuitive to ensuring signals are 
transmitted quickly. Still, it is essential to ensure that action potentials travel 
in one direction down the axon and helps regulate the frequency and timing of 
neuron firing. 


Action 
potential 
+40 


Voltage (mV) 
(je) 


=a0 
-70 


0 1 2 3 4 5 
Time (ms) 


Figure 1.3: Diagram of Action Potential [5] 
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The rise and fall of the influx or efflux of specific ions during the action 
potential are driven not only by the opening and closing of the channel but also 
by the ion’s equilibrium potential. The equilibrium potential of an ion is the 
membrane potential at which the net flow of that ion across the membrane is 
zero. The concentration gradient and the electrical gradient across the 
membrane determine it. When the membrane potential is equal to the 
equilibrium potential of a particular ion, there is no net movement of that ion 
across the membrane. Therefore, during the action potential, as the membrane 
potential changes, ions move in or out of the cell depending on whether the 
membrane potential is above or below their respective equilibrium potentials. 
This movement of ions contributes to the rise and fall of the action potential, 
allowing for the transmission of electrical signals along the neuron. The Nernst 
Equation can determine the equilibrium potential of the ions for a given ion 
across a semipermeable membrane[7]. The Nernst equation is expressed as: 


RT [ionsoutl 


Eion = (1.1) 


ZF [ions;n | 


Where Eion is the equilibrium or Nernst potential of the ion in volts, R is the 
ideal gas constant (8.314 J/Kmol), T is temperature in Kelvin, z is the charge 
of the ion, F is the Faraday constant (96, 485 A/mol), and [ionsou] and [ionsin] 
are the concentrations of ions outside and inside the cell, typically measured 
in millimoles per liter (mm/L). The Nernst equation is relevant for determining 
neurons' resting potential and action potentials’ generation. During the action 
potential, changes in the permeability of the ions across the membrane lead to 
shifts in the membrane potential. This equation can help predict the membrane 
potential when each ion will be in equilibrium during different phases. This 
helps to provide insights into mechanisms of neuronal excitability and signal 
propagation. However, the Nernst equation can only determine the membrane 
potential based on a single ion species. The Goldman Equation determines the 
permeability of multiple ions and their concentration gradients. This equation 
describes the resting membrane potential of a cell based on the permeability 
of multiple ions and 
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is expressed as: 

Where Vm is the membrane potential, R, T, and F represent the same 
parameters as the Nernst equation, Px’, Pna’, and Pcr are the permeability of 
the membrane to that ion, and [K*], [Na‘], and [Cl-] are the concentration of 
the ion. The Goldman Equation provides a more accurate representation of the 
membrane potential since it considers multiple ions with different permeability 
and accounts for complex interactions between ions and how they influence 
the membrane potential. 


RT . Pxtt [K* lout + Pat + [Na* ]out + Poi- — [Cl Jour 
Pxt + [K*]in + Pnat + [Na*]in at Poi- = [Clin 


(1.2) 
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Figure 1.4: The Cable Model [10] 
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Action Potential Propagation 


After the action potential is initiated at the axon hillock, the adjacent region of 
the axon's membrane is then depolarized, initiating another action potential. 
This sequential depolarization and repolarization occur along the entire length 
of the axon, propagating the action potential. 

Section 1.1 of this chapter discussed saltatory conduction and how this signal 
propagates by jumping from one node of Ranvier to the next. Several factors 
can influence the speed of the signal’s propagation down the axon, such as 
myelination, the diameter of the axon, and temperature. As the diameter of the 
axon gets larger, it offers less resistance to ion flow and will produce faster 
conduction velocities compared to smaller-diameter axons. Higher 
temperatures will also increase the speed of the action potential propagation 
by accelerating the rate at which the ion channels open and close [8]. Just as 
we can use the Nernst equation to understand the equilibrium potentials of ions 
computationally, we can use the Cable Theory and Model to mathematically 
represent and understand signal propagation along the axon. 


Cable Theory and Model 


OV ie ok av? 1 ov 
Ot Cm OX2 Mir Ox (1.3) 


The Cable Theory is a framework for understanding how the action potentials' 
electrical signals propagate along the axon's length. 

This theory models the neuronal axon as a cylindrical cable with electrical 
properties that govern signal transmission. The theory also highlights that 
space and time play a role in determining the speed and attenuation of signal 
propagation along an axon [9]. The theory derives the cable model (Fig. 1.4 
[10]) and the cable equation, which is expressed as: 

Where V is the membrane potential in volts as a function of position, x, along 
the neuron, and time, t, terms rm and Cm are the membrane resistance and 
capacitance per unit area, respectively. Term 7, is the longitudinal resistance 
per unit length. 
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Term 

av? 

Ox? 
is the rate of change of the membrane potential concerning time and is the 
second derivative of the membrane potential with respect to position. This 
equation describes how the membrane potential changes as the signal 
propagates down the axon over space and time. The equation accounts for the 
neuron's electrical properties and ion flow's longitudinal resistance. The cable 
model determines and predicts the spatial (space/position) and temporal (time) 
dynamics of action potential propagation. 
Now that we have understood how neurons can create action potentials and 
how those action potentials travel down the axon, we will explore how these 
concepts come together to generate neural communication. 


Neural Communication 


Once the electrical signal travels down the axon, it will reach the axon terminal 
containing synapses. At these synapses, the neuron can either communicate 
using an electrical synapse or a chemical synapse. 


Synaptic Transmission 


Electrical synapses, or gap junctions, connect adjacent neurons that allow for 
direct electrical communication. In these synapses, the electrical signal is 
transmitted through direct electrical coupling through gap junction channels 
formed by connexin proteins. These proteins span the membranes of the two 
neurons, creating a direct pathway from one neuron to the next. Gap junctions 
provide a fast and efficient way to transmit signals between neurons and are 
involved in sensory processing and motor control. 

On the other hand, a chemical synapse involves a more specialized junction 
between the neurons, which transmit signals through the release of chemical 
messengers called neurotransmitters. These synapses have several key 
components that are all important for proper signal transmission. The first 
component is the presynaptic terminal. This is where the signal has already 
traveled down the axon of a presynaptic neuron and is ready to send the signal 
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to the next neuron. The presynaptic terminal contains synaptic vesicles filled 
with neurotransmitters and voltage-gated calcium channels on its membrane. 
In between the two neurons is a narrow space called the synaptic cleft. The 
cleft acts as a physical barrier that neurotransmitters must cross to transmit 
signals from the presynaptic to the postsynaptic neuron. The neurotransmitters 
bind to protein receptors on the postsynaptic neuron. Chemical synapse 
transmission occurs by the following steps: 

1. The presynaptic neuron receives an action potential at the axon 

terminal, depolarizing the membrane. 
2. Voltage-gated calcium channels in the membrane open. 


iss) 


Calcium ions influx into the presynaptic neuron. 

4. Calcium binds to the synaptic protein synaptotagmin that anchors the 
vesicle to the membranes. 

5. The Calcium-bound synaptotagmin interacts with SNARE proteins on 
the vesicle and presynaptic membrane. This causes a conformational 
change in the SNARE complex. 

6. The vesicles, filled with neurotransmitters, will move toward and dock 
at the presynaptic membrane. 

a. The vesicles undergo exocytosis at the presynaptic membrane 
and release neurotransmitters into the synaptic cleft. 

b. Neurotransmitters diffuse across the synaptic cleft and bind to 
protein receptors on the postsynaptic neuron. 

c. The receptor undergoes a conformation change, leading to the 
opening or closing of ion channels and changes in the 
postsynaptic membrane potential. 

Chemical synapses are dynamic and plastic, allowing fine-tuning signal 
transmission in response to environmental stimuli [11]. They are essential for 
various nervous system functions, such as sensory processing, motor control, 
and memory. Many different neurotransmitters and receptors have roles in 
orchestrating signal transitions across chemical synapses. 
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Neurotransmitter and Receptors 


Neurotransmitters are chemical messengers that transmit signals across the 
chemical synapse and influence the activity of the postsynaptic neuron. 
Neurotransmitters interact with their specific receptors on the postsynaptic 
membrane to trigger changes in the membrane potential to either excite or 
inhibit neuron activity. When a neurotransmitter excites a neuron, it will 
deliver an excitatory postsynaptic potential (EPSP) that will depolarize the 
neuron, bringing it closer to the threshold to trigger an action potential. When 
a neurotransmitter inhibits a neuron, it will deliver an inhibitory postsynaptic 
potential (IPSP) that will hyperpolarize the neuron, bringing it closer to resting 
potential and making it harder to trigger an action potential [6]. When signals 
are delivered rapidly from one presynaptic neuron, the signals are summated 
in a temporal summation process. If signals are supplied simultaneously but 
from different presynaptic neurons, the signals will still summate in a process 
called spatial summation. Several types of neurotransmitters and _ their 
receptors deliver either EPSP or IPSP. Here, we will discuss the most common 
ones. 


Glutamate 

Glutamate is the primary excitatory neurotransmitter in our central nervous 
system (CNS) and plays a critical role in synaptic transmission, learning, and 
memory. The two glutamatergic (meaning a receptor or system that involves 
glutamate) are the AMPA (a-amino-3-hydroxy-5-methyl-4-isoxazolepropionic 
acid) and NMDA (N-methy1-D-aspartate) receptors. 


GABA (Gamma-Aminobutyric Acid) 


GABA is the primary inhibitory neurotransmitter in the CNS, mediating 
synaptic inhibition and regulating neuronal excitability. Its receptors are 
GABA-A and GABA-B. 

Dopamine 

Dopamine is critical in several processes, such as reward-motivated behavior, 
motor control, and cognition. Depending on its receptor subtype, it can deliver 
either EPSPs or IPSPs. For instance, the D1-like receptors, which are receptors 
D1 and D5, are typically excitatory and often associated with reward, 
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motivation, and motor control. D2-like receptors, which are receptors D2, D3, 
and D4, can deliver either signal type depending on where they are in the brain 
and under the context in which they are being used. 


Serotonin 

Serotonin effects can vary widely depending on the receptor used and the brain 
region in which it is used. Some of its receptors mediate EPSPs and some 
IPSPs. Serotonin is crucial in mood, appetite, sleep, and other physiological 
processes. Its receptors are the 5-HT receptors. 


Acetylcholine 

Acetylcholine's effects are also diverse depending on the receptor and location. 
In the CNS, it acts primarily as an excitatory neurotransmitter, while in the 
peripheral nervous system (PNS), it can deliver EPSPs or IPSPs. Its receptors 
are the nicotinic and muscarinic receptors. 

Now that we have established the different types of neural signal transmission 
and explored the various neurotransmitters contributing to that transmission, 
we will turn to the nervous system's boundary organization and function, 
where signal transmission integrates to produce our complex behaviors and 
physiological processes. 


The Nervous System 


The nervous system comprises two main components: the central nervous 
system (CNS) and the peripheral nervous system (PNS). Each plays a distinct 
but interconnected role in regulating and coordinating all bodily functions and 
responses. 


Central Nervous System (CNS) and Peripheral Nervous 
System (PNS) 


The CNS holds the brain and spinal cord and is the main control center for 
processing and interpreting sensory information, making decisions, and 
sending commands. The brain is the most complex organ in our bodies and is 
responsible for everything from simple things like our hearts beating to 
complex thoughts and emotions. The brain consists of various regions, such as 
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the brainstem, cerebellum, diencephalon, and cerebrum, each with different 
functions and responsibilities (Fig. 1.5 [12], [13]). 

Brainstem 

The brainstem consists of the midbrain, pons, and medulla oblongata and 
connects the brain to the spinal cord. It also controls vital functions such as 
heart, breathing rate, and blood pressure. 


Cerebellum 


The cerebellum is beneath the cerebrum and is crucial for balance, 
coordination, and voluntary movements. It ensures that our movements are 
smooth and precise. 


Diencephaton 


The diencephalon is a region that includes the thalamus and the hypothalamus. 
The thalamus is the relay station that transmits sensory and motor signals to 
the cerebrum to regulate sleep and alertness. The hypothalamus regulates 


Frontal lobe 


Temporal lobe 


Figure 1.5: The Four Subdivisions of the Brain (left) and the Regions of the Brain (right) 
[12],[13] 
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hunger, thirst, temperature control, and circadian rhythm. It also controls the 
pituitary gland. 


Cerebrum 


The cerebrum is the largest part of the brain and is responsible for higher 
functioning, such as thoughts, memories, emotions, and voluntary movements. 
It is divided into two hemispheres, which are divided into four lobes: the 
frontal, parietal, temporal, and occipital lobes. It is made up of both gray and 
white matter. The white matter primarily comprises the myelinated axons of 
the neurons, while the gray matter is composed of the soma, dendrites, 
unmyelinated axons, glial cells, and capillaries. 

The spinal cord acts as a conduit for signals between the brain and the rest of 
the body and is involved in reflexes and execution of simple motor commands. 
Itis a cylindrical structure that extends from the base of the brain down through 
the vertebral column. It is the main pathway for transmitting information 
between the brain and the rest of the body and is organized into distinct regions 
and structures (Fig. 1.6 [14]). 


Gray Matter 


The gray matter is in the center of the spinal cord and contains neuron cell 
bodies, dendrites, and unmyelinated axons. It is organized in the dorsal, 
ventral, and lateral horns. 
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White Matter 

The white matter surrounds the gray 
matter and consists of myelinated 
axons forming ascending and 
descending tracts. These tracts allow 


communication between the brain 
and the different parts of the spinal 
cord. 


Spinal nerves 


Thirty-one pairs of spinal nerves 


emerge from the cord and carry 
Figure 1.6: Grey and White Matter of the 


sensory information from the CNS to Spinal Cord [14] 


motor command to the peripheral 

tissues. 

The structure of the CNS allows for vast amounts of information to be 
processed as fast and as efficiently as possible to coordinate behaviors and 
maintain vital physiological processes. This organization is needed for the 
proper functioning of the nervous system. 

The PNS connects the CNS to the rest of the body and is divided into two 
subdivisions: the somatic and autonomic nervous systems. The somatic 
nervous system controls our voluntary movements by sending motor 
commands from our CNS to the skeletal muscles. It also sends sensory 
information from the skin, muscles, and sensory organs to the CNS to perceive 
touch, pain, temperature, and body position. The autonomic nervous system 
regulates involuntary functions such as heart rate, digestion, respiratory rate, 
and blood pressure. This system is further divided into two branches: the 
sympathetic and parasympathetic nervous systems. The sympathetic nervous 
system controls our fight or flight response to prepare our bodies for stress or 
emergencies. The parasympathetic nervous system controls our rest and 
digestion response to help maintain baseline conditions or homeostasis. 

The CNS and PNS allow the body to interact and respond to the environment, 
ensuring we can function optimally in a complex network of sensory inputs, 
integrative processes, and command outputs. 
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Chapter 1: Summary 


In this first chapter, you have embarked on the beginning of your journey to 
the world of neural engineering, beginning with an exploration of neurons, the 


building blocks of the nervous system. We uncovered the diversity and 
complexity of neurons by discussing their morphology, functionality, and 
mechanisms for neural communication. Then, we explored synaptic 
transmission and the nervous system's organization to understand better the 
complexity underlying our every sensation, thought, and action. We have laid 
a solid foundation for your next journey in Chapter 2, exploring the Hodgkin- 
Huxley model. In this chapter, we will dive deeper into the principles 
governing neuron excitability, paving the way for understanding neural 
functioning and the development of innovative technologies aimed at 
interfacing the nervous system. 
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Chapter 1: Learning activities 


Learning Activity 1.1 


Exploring Action Potentials Through 
Simulations 


Objective 


To understand the characteristics and dynamics of action potentials by 
comparing different online simulations. 


Materials Needed 
e Computers or tablets with internet access 
e Projector or whiteboard for class discussion 
e Notebooks or digital note-taking tools 


Instruction: 


Step 1: Individual Exploration 
Find and explore an online simulation of an action potential. Some popular 
options might include: 
e The "NeuroLab Action Potential Simulator" by the University of Utah 
e "Interactive Neuron Model" on the HHMI Biointeractive website 
e "Action Potential Simulator" on PhET Interactive Simulations 
Spend about 15-20 minutes interacting with the simulation. Take notes on the 
key features, such as the action potential phases (depolarization, 
repolarization, hyperpolarization), the role of ion channels, and the overall 
shape of the action potential graph. 


Step 2: Group Discussion 

Pair up with another student and share your findings. Discuss the following: 
e Commonalities: What similarities did you notice between the two 
simulations? Consider aspects like the basic shape of the action potential, 
the phases (depolarization, repolarization, hyperpolarization), and the role 
of ion channels. 
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e Differences: What differences did you observe? Consider variations in 
the simulation interface, the level of detail provided, and any differences 
in how ion channel activity is depicted. 
Discuss and make note of the explanations provided by each simulation for the 
phenomena occurring during an action potential. For instance, how does each 
simulation explain the opening and closing of sodium and potassium channels? 


Step 3: Class Discussion 
Reconvene as a class and present your findings. Each pair should address the 
following: 
e Summarize the common aspects observed in the simulations. 
e Highlight and explain the differences you found. 
e Describe the phenomena of the action potential as observed in the 
simulations. Include explanations for: 
e Depolarization: How and why the membrane potential rises. 
e Repolarization: The process of returning to the resting 
membrane potential. 
e Hyperpolarization: Why the membrane potential 
temporarily becomes more negative than the resting potential. 
e Jon Channel Activity: The role of sodium and potassium 
channels in the action potential phases. 
e Be prepared to discuss any questions or clarifications that arise from 
the class discussion. 
Debrief: 
e Conclude with a summary of key points learned from the activity. 
e Reflect on how simulations can aid in understanding complex biological 
processes like action potentials. 
e Discuss any insights gained about the similarities and differences between 
different simulation tools. 
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Learning Activity 1.2 


Buzz groups: Block diagram of the nervous 
system 10min 


Buzz groups are a collaborative learning strategy used in 
educational and training settings to facilitate discussion and 
idea generation among participants. The technique involves dividing a large 
group into smaller subgroups, typically consisting of 4 to 6 members, who then 
engage in focused discussions on a specific topic or question for a short period 
of time, usually around 10-15 minutes. The name "buzz groups" comes from 
the lively and energetic nature of these small-group discussions, which can 
often sound like a buzzing noise when many groups talk simultaneously. 


Key Features of Buzz Groups: 


e Small Size: Each buzz group is kept small to ensure that every 
member has an opportunity to contribute to the discussion. 

e Focused Discussion: Groups are given a specific question or topic to 
discuss, which helps to keep the conversation on track and 
productive. 

e Time-Limited: Discussions are typically short, creating a sense of 
urgency and encouraging participants to share their ideas quickly and 
concisely. 

e Reporting Back: After the discussion, each buzz group usually 
reports their findings, ideas, or questions back to the larger group, 
facilitating a broader exchange of ideas. 

e Facilitation: An instructor or facilitator may guide the process by 
providing the discussion topic, setting the time limit, and helping to 
synthesize and integrate the groups' findings. 


Benefits of Buzz Groups: 
e Increased Participation: By breaking a large group into smaller units, 
more participants will likely speak up and engage in the discussion. 
e Diverse Perspectives: Buzz groups allow for a variety of viewpoints 
to be expressed and considered, leading to richer and more 
comprehensive discussions. 
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Active Learning: Participants are actively involved in the learning 
process, which can enhance understanding and retention of the 
material. 

Critical Thinking: The format encourages participants to think 
critically and creatively while discussing and analyzing the topic. 
Collaboration Skills: Working in small groups helps participants 
develop and practice collaboration and communication skills. 


Phase 1: Create random groups of 2-3 people. Propose the following question. 
Draw a block diagram describing the functional behavior of the nervous 
system. Give 10 to 15 min, encourage students not to look at internet resources 


initially and to try to reason their diagram. 


Phase 2: report the block diagram to the rest of the class and share. 


Inclusivity aspect of the class learning activity: 


Collaboration: tasks are given in teams, and technical discussions are 
expected. 

Representation: Encourage the groups to integrate all members into 
the team. For example, everybody should hold a pen and draw the 
diagram at approximately the same time. When discussing the topic, 
all students should have approximately the same time to talk; in other 
words, everybody is heard and respected. 

Autonomy: students get to pick the type of block diagram, which 
engineering aspects of the nervous system they want to be represented, 
etc. 

Safe space: this activity is not graded. Through classroom discussion, 
students will be helped to complement their diagrams. 
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Learning Activity 1.3 


Understanding Action Potentials Through 
Interactive Gameplay 


Objective 
To deepen understanding of the action potential process by playing an 
interactive game and discussing its various components and challenges. 


Materials Needed 

e Computers or tablets with internet access 

e Access to the game: Action Potential in Neuron Game 
e Notebooks or digital note-taking tools 


Instructions 


Step 1: Individual Gameplay 

e Access the "Action Potential in Neuron" game using the provided link. 

e Play the game individually to familiarize yourself with its mechanics and 
objectives. 

e Take note of your scores and the time it took to complete the game. 

e Record any aspects of the game that you found particularly challenging or 
confusing, especially in relation to the action potential process. 


Step 2: Group Discussion 
Form small groups of 3-4 students and share your individual experiences with 
the game. 

e Scores and Timing: Briefly report your scores and completion times to each 
other. Discuss if there were any noticeable differences among group 
members. 

e Problematic Aspects: Identify and discuss which parts of the game were 
the most challenging. Focus on specific aspects of the action potential that 
you found difficult to understand or replicate in the game. 

e Understanding the Action Potential: Reflect on how the game represented 
different phases of the action potential, such as depolarization, 
repolarization, and hyperpolarization. Discuss whether the game 
effectively illustrated these concepts. 
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e Need for Clarification: Highlight any aspects of the action potential that 


need further explanation or were unclear in the game. Consider whether the 
game provided adequate information about ion channels, membrane 
potential changes, or other key elements. 


Step 3: Class Discussion 


Reconvene as a class and share your group's findings. 


Common Challenges: Summarize the common difficulties faced by 
different groups. Discuss why certain aspects of the game were 
challenging and how they relate to the action potential process. 

Game Effectiveness: Evaluate the game's effectiveness in teaching the 
concepts of action potentials. Were there any elements of the game that 
were particularly helpful or misleading? 

Further Explanation: Based on the group discussions, identify any parts of 
the action potential process that need additional clarification. Be prepared 
to explain these concepts using diagrams or additional resources if needed. 


Debrief: 


e Summarize the key takeaways from the activity, emphasizing the 
insights gained from both playing the game and discussing its content. 

e Reflect on how interactive tools like games can _ enhance 
understanding of complex biological processes and where they might 
fall short. 

e Discuss strategies for using similar interactive tools to improve 
learning in other areas of biology or science. 

@ 


rr 
PAs 


43 


Chapter 1: Lab introduction 


In this series of lab exercises, you will explore the fundamental 
principles of neuroscience through practical applications using 
advanced engineering tools. These labs are designed to provide 
hands-on experience with MATLAB and Arduino, enabling you to 
deepen your understanding of neuron function, morphology, and 
simulation. You will begin by studying the structure and 
communication mechanisms of neurons, the brain's fundamental 
building blocks. 

Using MATLAB and the TREES Toolbox, you will visualize and 
analyze the morphology of different neurons. This toolbox allows 
for comprehensive analysis and visualization of neuron structures, 
providing insights into how their shapes and forms influence their 
functions. By working with the provided datasets, you will generate 
visualizations to explore and compare the morphological differences 


between various neuron types. 


In the subsequent lab, you will simulate the behavior of synthetic 
neurons using an Arduino microcontroller. This simulation will be 
based on the Hodgkin-Huxley model, a foundational mathematical 
model that describes how action potentials in neurons are generated 
and propagated through the movement of ions across the cell 
membrane. Throughout these labs, you will apply theoretical 
knowledge to practical scenarios, preparing you for further studies 
in neuroscience, biomedical engineering, and related fields. 
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Chapter 1: Lab Example 1 


Introduction 


This first chapter explains the fundamental principles of the 
brain, including the neurons, and how they communicate 


with one another to execute various functions throughout 

the body. We will apply what we have discussed to a practical engineering tool. 
In this first lab example, we will use MATLAB. This platform will provide the 
tools for simulating and analyzing the differences in morphology of different 
neurons using the TREES Toolbox. This toolbox is comprehensively designed 
to analyze and visualize neuron morphology structures. In this lab example, 
we will download the toolbox and generate visuals using the dataset provided 
to look at the morphology of different neurons. 


Steps 


First, you will need to download MATLAB if you do not already have it; it 
will become useful in the neural engineering journey in this textbook and in 
many other instances of your engineering journey. Follow this link to 
download MATLAB 


https://www.mathworks.com/products/matlab/student.html 


After downloading MATLAB to your device, download the TREES toolbox 
using this link. 


https://www.mathworks.com/matlabcentral/fileexchange/68886-trees-toolbox 
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Figure 1.7: Pop up box 


Once on the website, you will download the current version of the TREES 
toolbox from GitHub. The toolbox comes in the form of a zip file (Fig 1.7). 
When the zip file is downloaded onto your device, unzip the toolbox. Other 
toolboxes recommended for the TREES Toolbox are the Image Processing 
Toolbox, in polyhedron, and the Statistics and Machine Learning Toolbox, 
which can be added using the MATLAB Add-On Explorer. 

Now that we have the toolbox, we can explore it in MATLAB. The first step 
is to unzip the TREES toolbox file and then add the TREES Toolbox to your 


Figure 1.8: Red error message and successful upload message 


46 


MATLAB path. You do this by importing the file using the import folder 
button. Then right-click the file in the Current Folder box and select Add to 
Path. Once in your MATLAB path, code in >>start_trees in the command 
window. Another way to start the toolbox is by navigating to the start_trees.m 
file and clicking run. You will get the red error message if you have 
unsuccessfully uploaded the trees toolbox (Fig. 1.8). You will see the second 
message in black if you have successfully uploaded it. 

Now that the toolbox is loaded into your MATLAB, we will visualize neuron 
morphology. 

First, go to the command window and code >>load_tree. Once you have done 
that, MATLAB will open the Pick a file pop-up box (Fig. 1.9). Navigate to the 
samples folder, then to the mtr folder. The TREES toolbox saves the 
morphology files in. mtr format. We will select the sample .mtr file for this lab 
and click Open. 

Next, to upload the sample as a visual image, code >>fix_tree_UI in the 
command window. You will be prompted with a pop-up box asking you to 
upload an input tree to be repaired. Click OK and navigate back to the “sample. 
mtr’ file. You will then be prompted to upload a reference tree and navigate to 
the same “sample. mtr” file. Once you have uploaded the input and reference 
tree, a “fix_tree_UI” pop-up window will open, showcasing the sample. 


Figure 1.9: Results of >>>load_trees command 


47 


Figure 1.10: Trees upload result 


Can you use your knowledge from this chapter to know what morphological 
structure is depicted in Fig. 1.10? If you said axon terminals, you are correct! 
In this sample, we see a visual representation of the axon terminals and a 
potential number of synaptic connections this sample could make. 

Now, we will upload a new sample. Follow the same steps for the “sample. 
mtr” file, but choose the “hss.mtr” file instead. Remember to upload this as 
your tree to be repaired and your reference tree. 

This file shows an example of an entire neuron showcasing all the 
morphological features (Fig. 1.11). You can use the zoom-in and zoom-out 
buttons in the top left corner to take a closer look at the different parts of the 
neuron. We can take this image rending one step further by uploading an image 
stack. 

Navigate to the Load stack button in the right-hand corner. The “Pick a file” 
pop-up menu will open. Navigate to the test, stacks, and data folders, and click 
the available stack image. 
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Figure 1.11: Visual of the entire neuron 


Figure 1.12: Entire neuron and its connections 


In this new image rendering, we can see how the morphology of these neurons 
is situated and what other neurons around it look like (Fig. 1.12). This toolbox 
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can also visualize neurons, measure dendritic length, branch points, and 
surface area of neurons, and conduct statistical analysis on these parameters. 
We encourage you to continue exploring this toolbox to perform some of your 
analyses on the neuron samples provided. 


Other Examples and Resources 


Several other examples are available for you to use to keep exploring this 
toolbox. One resource type is Hermann Cuntz's free TREES toolbox tutorials 
on YouTube, which can be accessed using this link. 


https://www. youtube.com/@manodelcatan 


For more examples of practical applications of this chapter's content, please 
visit the book's dedicated GitHub repository page using this link. 


https://github.com/bddupre92/Neurobook BME UND 
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Chapter 1: Lab Example 2 


Synthetic Neuron Simulation with Arduino 


This repository contains the code and instructions for 


simulating the behavior of a synthetic neuron using an 
Arduino microcontroller. The simulation is based on the Hodgkin-Huxley 
model, which describes how action potentials in neurons are initiated and 
propagated through the movement of ions across the cell membrane. 


Lab Overview 


In this lab, you will: 1. Understand the Hodgkin-Huxley model and its 
application in simulating neuronal behavior. 2. Implement the Hodgkin- 
Huxley equations in Arduino code. 3. Simulate the behavior of a synthetic 
neuron and observe the output. 4. Use an oscilloscope to visualize the 
membrane potential signal. 


Materials Needed 


e Arduino microcontroller 

e Breadboard and connecting wires 

e Resistors and LEDs (optional for visual representation of the output) 
e Computer with Arduino IDE installed 

e Oscilloscope 


Figure 1.12: Initial hardware setup for the Arduino-based synthetic neuron 
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Key Concepts 
Hodgkin-Huxley Model 
The Hodgkin-Huxley model is a set of nonlinear differential equations that 
describe the ionic mechanisms underlying the initiation and propagation of 
action potentials in neurons. The model includes variables for membrane 
potential (V) and gating variables (n, m, and h), representing the probability of 
open or closed ion channels. 
e Membrane Potential (V): The voltage difference across the neuron’s 
membrane. 
e Gating Variables (n, m, h): Variables representing the state of 
potassium (n), sodium activation (m), and sodium inactivation (h) 
gates. 


Equations 
The following vital equations define the model: 
1. Membrane Potential (V): Describes the change in membrane 
potential over time-based on ion conductance and injected current. 
2. Gating Variables (n, m, h): Describe the dynamics of the gating 
variables that affect ion channel conductance. 


Procedure 


Setup the Arduino Environment 


1. Connect the Arduino to your computer. 
2. Open the Arduino IDE and create a new sketch. 


Initialize Variables and Constants 

1. Define initial values for the gating variables (n1, m1, h1) and 
membrane potential (V1). 

2. Define constants such as membrane capacitance (C), maximum 
conductance for potassium (g_k_max), sodium (g_Na_max), and leak 
channels (g_L), and equilibrium potentials for potassium (E_K), 
sodium (E_Na), and leak channels (E_L). 
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Implement the Hodgkin-Huxley Equations 
1. Write functions to calculate the derivatives of the gating variables 
(n_prime, m_prime, h_prime). 
2. Update the state of the neuron in the loop function by calculating new 
values for the gating variables and membrane potential. 
Output the Membrane Potential 
1. Use the analogWrite function to output the membrane potential to 
pin 3 on the Arduino. 
2. Connect pin 3 to the oscilloscope to visualize the signal. 
Connecting the Oscilloscope 
1. Connect the ground probe of the oscilloscope to the ground pin on the 
Arduino. 
Connect the signal probe of the oscilloscope to pin 3 of the Arduino. 
3. Turn on the oscilloscope and set it to the appropriate voltage and time 
scales to visualize the signal from the Arduino. 


Arduino Code 


// Synthetic Arduino neuron simulation 


// Initial values for gating variables and membrane 


potential 

double ni = 0.00@3; 
double mi = @.0011; 
double hi = 0.9998; 


double V1 = -1@; 


// Constants for the neuron model 


double C = 1; // Membrane capacitance 
double g k_max = 36; // Maximum conductance for 
potassium channels 

double g Na_max = 120; // Maximum conductance for 
sodium channels 

double g L = 0.3; // Conductance for leak 
channels 
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double E_K = 
potential 
double E_Na 
potential 
double E_L = 
potential 
double dt = 
simulation 
double I_inj 


// Nariables 
double n, mo, 


-12; // Potassium equilibrium 


= false // Sodium equilibrium 
10.613; // Leak equilibrium 
@.04; // Time step for the 
= 10; // Injected current 


to store the current state of the neuron 
h, V5 


double n_new, m_new, h_new, V_new; 


void setup() { 
// Setup code to run once 
pinMode(3, OUTPUT); // Set pin 3 as output 
Serial .begin(960@) ; // Begin serial 


communication at 9600 baud rate 


// Initial 
n=n1; 
m = m1; 
h = hi; 
V=V1; 


si 


void loop() 


ize the state variables 


i 


// Main loop for the simulation 


// Calcula 
double dn 
double dm 
double dh 


// Update 
n_new = n 
m_new = m 
h_new = h 


te the derivatives of the gating variables 
= n_prime(n, -V); 
= m_prime(m, -V); 
= h_prime(h, -V); 


the gating variables 
+ dn * dt; 
+ dm * dt; 
+ dh * dt; 
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// Calculate the change in membrane potential 
double dV = 1 / C * (-1 * g k_max * pow(n, 4) * (V - 
E_K) 
- g Na_max * pow(m, 3) * h * (V - 
E Na) 
- gL * (V - E_L) + I_inj); 
V_new = V + dV * dt; 


// Output the updated membrane potential to pin 3 
analogWrite(3, V_new + 20); 


// Update the state variables for the next iteration 
n = n_new; 
m = m_new; 
h = h_new; 
V = V_new; 


} 


// Function to calculate the derivative of the n-gating 
variable 
double n_prime(double n, double V) { 

double k_1 = (@.01 * (V + 10)) / (exp((V + 10) / 10) - 
1); 

double k_2 = @.125 * exp(V / 8@); 

double dn = k_1 - (k_1 + k_2) * n; 

return dn; 


} 


// Function to calculate the derivative of the m gating 
variable 
double m_prime(double m, double V) { 

double k_1 = @.1 * (V + 25) / (exp((V + 25) / 10) - 
1); 

double k_2 = 4 * exp(V / 18); 

double dm = k_1 - (k_1 + k_2) * m; 

return dm; 


} 
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// Function to calculate the derivative of the h gating 
variable 
double h_prime(double h, double V) { 
double k_1 = @.@7 * exp(V / 20); 
= iy (expC (V 430) 7/10) 1), 
double dh = k_1 - (k_1 + k_2) * h; 
return dh 
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Figure 1.12: Arduino-based neuron firing 


Synthetic neuron extended: Neuron Simulation with Ad- 
justable Threshold 

Another extension to this lab can be to adjust some of the variables reading the 
ADC from a potentiometer. 


Overview 

This extension of the previous lab enhances the synthetic neuron simulation 
lab by adding a potentiometer to adjust the action potential threshold 
dynamically. The simulation is based on the Hodgkin-Huxley model, which 
describes the initiation and propagation of action potentials in neurons. 
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Objectives 


1. 
2, 


Integrate a potentiometer into the Arduino setup. 

Modify the code to read the potentiometer value and adjust the action 
potential threshold (I_inj). 

Using an oscilloscope, visualize the effects of changing the threshold 
on the membrane potential. 


Materials Needed 


Setup 


Arduino microcontroller 

Breadboard and connecting wires 

Potentiometer (10kQ recommended) 

Resistors and LEDs (optional for visual representation of the output) 
Computer with Arduino IDE installed 

Oscilloscope 


Connecting the Potentiometer 


if 


Connect one outer pin of the potentiometer to the 5V pin on the 
Arduino. 


Figure 1.12: Initial hardware setup of Arduino with potentiometer 
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2. Connect the other outer pin to the GND pin. 
3. Connect the middle pin (wiper) to an analog input pin on the Arduino 
(e.g., AO). 


Arduino Code snippet 

void loop() { 
// Read the potentiometer value (@-1023) 
int potValue = analogRead(A@); 


// Map the potentiometer value to a suitable range for 
If] (Eofian @) 10) 2) 
double I_inj = map(potValue, @, 1023, 0, 20); 


Figure 1.12: Arduino-based neuron firing 


58 


59 


Chapter 2 


Hodgkin-Huxley Model, 
Mathematical Modeling of 
Neurons 


Introduction and Learning Objectives 


In this chapter, we will explore the critical role of the Hodgkin-Huxley model 
in neuroscience. This model has been fundamental in understanding how 
neurons generate and propagate electrical signals. The chapter will provide a 
comprehensive understanding of the historical impact, experimental setups, 
and broader applications of the Hodgkin-Huxley model. By the end of this 
chapter, you will be able to: 


Understand the historical impact of the Hodgkin-Huxley model and 
how it has revolutionized neuroscience. 

Analyze the vital experimental setups and data that led to the 
development of the Hodgkin-Huxley model. 

Learn how the Hodgkin-Huxley model can be generalized to 
understand diverse and complex neurons, including those in 


humans. 

Recognize the transformative effects of the Hodgkin-Huxley model 
on neuroscience research and applications. 

Demonstrate knowledge of the experimental techniques and data 
underpinning the development of the Hodgkin-Huxley model. 
Apply the principles of the Hodgkin-Huxley model to various types 
of neurons and understand its broader implications in neuroscience. 


Historical Context 


The Hodgkin-Huxley model has been crucial in advancing our understanding 
of neuronal behavior. By modeling the ionic mechanisms underlying the action 
potential, this model has provided insights into the functioning of neurons and 
laid the foundation for numerous advancements in neuroscience. 
Understanding the historical development, foundational experiments, and 
broad applications of the Hodgkin-Huxley model will equip you with a deeper 
appreciation of its significance and versatility. 
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Early Neuroscience: Foundations and Chal- 
lenges 


Figure 2.1: Ramon y Cajal in 1899 [15] 


Neuroscience as a distinct scientific 
field began in the late 19th and early 
20th This period 
significant advances in 


centuries. saw 
our 
understanding of the nervous system, 
driven by 


and 


primarily technological 


innovations groundbreaking 


theoretical work. Despite these 
advances, the mechanisms underlying 
neuronal signaling remained largely 
mysterious. 

One of the earliest figures to leave a 
mark on the field was Santiago Ramon y 
Cajal (Fig. 2.1 [15]), often called the 


father of modern neuroscience. Cajal’s 


studies of neural tissues using the Golgi staining method provided detailed 


insights into the structure of neurons. His work demonstrated that neurons are 


more discrete entities than a continuous network, which was previously 


believed under the reticular theory. This discovery laid the groundwork for the 


neuron doctrine, which states that 
the neuron is the fundamental unit 
of the nervous system [16]. 

Camillo Golgi, whose staining 
technique made Cajal’s discoveries 
possible, contributed 
significantly to early neuroscience 
(Fig 2.2 [17]). Although Golgi 
initially supported the reticular 


also 


theory, his invention of the silver 
nitrate staining method allowed for 


Figure 2.2: Camillo Golgi in his lab [17] 
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“negative Schwankung” 


Julius Bernstein (1839-1933) 


Figure 2.3: Bernstein and membrane theory permission pending [22] 


unprecedented visualization of neuronal structures. This technique, known as 
the Golgi stain, remains a pivotal tool in neuroanatomy [18], 


Additional Historical Figures 


e Charles Sherrington: His work on synapses and reflexes provided 
crucial insights into neuronal communication and integration [19]. 

e Edgar Adrian: Known for his work on the electrical activity of 
neurons, Adrian's discoveries about nerve impulses contributed to our 
understanding of how information is transmitted in the nervous system 
[20]. 

Despite these structural insights, neurons' electrophysiological properties were 
poorly understood. Early 20th-century neuroscientists faced substantial 
technical limitations. The tools available at the time, such as rudimentary 
electrodes and recording devices, were insufficient for detailed 
electrophysiological studies. Researchers could observe gross electrical 
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activity in the brain but lacked the precision to dissect the ionic mechanisms 
at play within individual neurons [21]. 


Theoretical Beginnings: Membrane Theory 


In the late 19th century, Julius Bernstein proposed the first significant theory 
regarding the electrical properties of neurons, known as the "membrane 
theory" (Fig. 2.3 [22]). Bernstein suggested that neurons generate electrical 
signals through changes in membrane permeability to ions. He postulated that 
the resting membrane potential of a neuron resulted from the differential 
distribution of ions across the cell membrane, with a higher concentration of 
potassium ions (K*) inside the cell and a higher concentration of sodium ions 
(Na*) outside [18]. 

While Bernstein's membrane theory was a crucial step forward, it lacked 
empirical support and detailed explanation. The theory did not account for the 
dynamic action potential generation and propagation processes. Moreover, the 
experimental techniques of the time were not advanced enough to test 
Bernstein's hypotheses rigorously. 


Technological Advances: Cathode Ray Oscilloscope 


The early 20th century saw the development of new technologies that would 
eventually enable more precise electrophysiological studies. One such 
advancement was Karl Ferdinand Braun's invention of the cathode ray 
oscilloscope. This device allowed for the visualization of electrical signals and 
gave researchers a powerful tool to study neuronal activity. The oscilloscope 
could display rapid changes in voltage, making it possible to observe the 
electrical behavior of neurons in real-time [20]. 

The oscilloscope, along with other technological innovations such as improved 
amplifiers and recording equipment, paved the way for more detailed 
investigations into the electrical properties of neurons. These tools were 
essential for the subsequent work of Hodgkin and Huxley, who would bring a 
new level of precision to studying neuronal electrophysiology [21]. 
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Other Early Experiments and Observations 


Before Hodgkin and Huxley's work, other researchers had significantly 
contributed to understanding neuronal electrical activity. Keith Lucas, a British 
physiologist, conducted early experiments on the electrical properties of nerve 
fibers. Lucas’s work on the conduction velocity of nerve impulses provided 
essential insights into the speed at which electrical signals travel along 
neurons. Although his studies were limited by the technology of the time, they 
laid the groundwork for future research. 

In the 1930s, Kenneth Cole developed the voltage clamp technique, which 
allowed researchers to control the membrane potential of a neuron and 
measure the resulting ionic currents. This breakthrough technique enabled 
precise ionic current measurements, contributing to action potential 
generation. Cole’s work on the electrical properties of cell membranes was 
instrumental in developing the Hodgkin-Huxley model [23]. 


The Hodgkin-Huxley Model 


Alan Hodgkin and Andrew Huxley's Pioneering Work 


In the early 1950s, Alan Hodgkin and Andrew Huxley (Fig. 2.4 [24]) embarked 
on a series of groundbreaking experiments that would fundamentally alter our 
understanding of how neurons generate and propagate electrical signals. Their 
work focused on the squid's 
giant axon, an experimental 
model chosen for its large 
size, facilitating precise 
electrophysiological 

measurements [25]. The 


insights gained from these 
experiments led to the 


Hodgkin-Huxley model, a ( 
set of mathematical 


equations that describe the 
ionic mechanisms Figure 2.4: Hodgkin and Huxley [24] 
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underlying action potentials. The choice of the squid’s giant axon as an 
experimental model was pivotal in advancing our understanding of neuronal 
function. The squid's giant axon, which can be up to | mm in diameter, 
provided a unique opportunity for detailed electrophysiological studies. Its 
large size made it easier to insert electrodes and precisely measure electrical 
activity. This feature was crucial for the experiments conducted by Hodgkin 
and Huxley, as it allowed them to perform detailed analyses that were 
impossible with smaller mammalian neurons [21]. One of the key 
technological advancements that enabled Hodgkin and Huxley's discoveries 
was Kenneth Cole's development of the voltage clamp technique in the 1930s. 
The voltage clamp allowed researchers to control the membrane potential of a 
neuron while measuring the ionic currents that flowed across the membrane 
[21]. This revolutionary technique separated the voltage control from the 
current measurement, providing a clear view of the ionic processes occurring 
during an action potential. 


Experimental Procedure and Key Findings 


Hodgkin and Huxley conducted their experiments by isolating the squid's giant 
axon and inserting electrodes to apply voltage clamps (Fig. 2.5 [26]). 

They varied the membrane potential and recorded the resulting ionic currents. 
Through these experiments, they discovered two main ionic currents 
responsible for the action potential: a rapidly activating inward current carried 
by sodium ions (Na*) and a more slowly activating outward current carried by 
potassium ions (K*) [25]. 
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Mathematical Modeling 


Hodgkin and Huxley went beyond merely describing their experimental 
observations; they developed a set of mathematical equations that 
quantitatively described the ionic mechanisms underlying action potentials. 
Understanding that conductance (denoted as g) represents the ease with which 
ions can flow through channels in the neuron's membrane is essential. 
Conductance depends on the number and state of ion channels open at any 
given time and can vary dynamically during the action potential. The model 
explicitly addresses the conductance of potassium (gx) and sodium (gna ) ions, 


Figure 2.5: Dissection of the Giant Squid Axon [26] 
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which play crucial roles in the generation and propagation of action potentials 
[23]. Their model included several vital components, also shown in Fig. 2.7 
[27]: 

e Sodium Conductance(gna): Describes the flow of sodium ions into the 
neuron, which is responsible for the initial rapid depolarization phase 
of the action potential. 

e Potassium Conductance (gx): Describes the flow of potassium ions out 
of the neuron, which is responsible for the repolarization phase that 
follows the peak of the action potential. 

e Leakage Conductance (gr): Accounts for the small, steady flow of ions 
not carried by voltage-gated channels, contributing to the resting 
membrane potential. 

e Membrane Capacitance (Cm): Represents the ability of the neuron’s 
membrane to store and separate charge [28]. 


Outside Im 


Figure 2.6: Hodgkin & Huxley Model Circuit Representation [27] 
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The core of the Hodgkin-Huxley model is expressed through a set of 
differential equations that describe how these conductances change over time 
and in response to changes in membrane potential. These equations provided 
a comprehensive framework for understanding the dynamics of action 
potential generation and propagation. 


dV 1 
ap = Una + lk + It + Text) (2.1) 
™. 


Where: 

e Vis the membrane potential. 

e Ina, Ix, and J, are the sodium, potassium, and leakage currents, 

respectively. 

e I.¢ is the externally applied current. 

e C,, is the membrane capacitance. 
The model also included equations for the gating variables m, h, and n, which 
describe the probability of ion channels being open or closed: 


dm 


PT Am (1 —m) — Bram (2.2) 
apo a,(1—h) — Byh (2.3) 
a 1 2.4 

Ge onl —n)— Byn (2.4) 


Where a and f are voltage-dependent rate constants. 
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Revolutionizing Neuroscience 


The Hodgkin-Huxley model was revolutionary for several reasons. First, it 
provided a quantitative description of the ionic basis of action potentials, 
allowing for precise predictions of neuronal behavior under various 
conditions. Second, it demonstrated the power of combining experimental data 
with mathematical modeling, setting a new standard for research in biophysics 
and neurophysiology [19]. 
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Figure 2.7: Hodkin & Huxley model Representation [27] 
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The publication of Hodgkin and Huxley's findings in a series of landmark 
papers in 1952 profoundly impacted the field of neuroscience. Their work 
explained how action potentials are generated and provided a framework that 
could be applied to other types of neurons and excitable cells. The Hodgkin- 


Normal Electroencephalogram (EEG) signal 


Figure 2.8: Healthy Brain Vs Brain with Seizures [29] 


Huxley model became a foundational tool in electrophysiology and 
computational neuroscience, influencing various research areas [21]. 
Applications and Extensions of the Model 


The Hodgkin-Huxley model has been widely applied and extended since its 
original formulation. Researchers have used the model to study various aspects 


of neuronal function, including: 
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e Neural Coding: Investigating how neurons encode and transmit 
information through patterns of action potentials. 

e Neural Plasticity: Exploring how changes in ion channel properties 
can affect synaptic plasticity and learning. 

e Disease Mechanisms: Understanding the ionic basis of neurological 
disorders such as epilepsy, where abnormal ion channel function leads 
to excessive neuronal excitability. 

e Pharmacology: Testing the effects of drugs on ion channel behavior 
and neuronal excitability, aiding in the development of new treatments 
for neurological conditions. 

Extensions of the Hodgkin-Huxley model have incorporated additional 
complexities, such as the role of calcium ions (Ca?*) in neuronal signaling, the 
effects of temperature on ion channel kinetics, and the integration of synaptic 
inputs from other neurons. 

For practical applications of this model, the studies below have utilized it in 
research on EEG, Epilepsy, and Behavioral Sciences (Fig 2.8 [29]). 

1. A Computational Model to Determine Membrane Ionic Conductance 
Using Electroencephalography in Epilepsy 

. The Hodgkin-Huxley Heritage: From Channels to Circuits 

3. Hodgkin and Huxley Opsin Model for Computationally Efficient 
Optogenetic Neurostimulation in Cells and Networks 

4. Action Potential Initiation in the Hodgkin-Huxley Model 


Continuing Impact and Future Directions 


The Hodgkin-Huxley model remains a cornerstone of neuroscience research. 
Its principles continue to inform the development of new models and 
experimental techniques. Advances in computational power and _high- 
throughput experimental methods have enabled the simulation of increasingly 
complex neural networks, building on the foundation of Hodgkin and Huxley. 
Future research aims to integrate the Hodgkin-Huxley model with new 
molecular biology and genetics insights, providing a more comprehensive 
understanding of neuronal function. For example, researchers are investigating 
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how genetic variations in ion channel genes contribute to individual 
differences in neuronal excitability and susceptibility to neurological diseases. 
The Hodgkin-Huxley model also serves as an educational tool, helping 
students and researchers grasp the fundamental principles of neuronal 
signaling. Its combination of experimental rigor and mathematical elegance 
inspires new generations of neuroscientists. 


The Hodgkin-Huxley Model's Lasting Influ- 
ence 


Revolutionizing Neuroscience 


The Hodgkin-Huxley model revolutionized neuroscience by providing a 
comprehensive and quantitative framework for understanding the electrical 
properties of neurons. Before their work, the mechanisms underlying action 
potentials were largely speculative, and the field lacked a unified theoretical 
foundation. The introduction of the Hodgkin-Huxley model marked a 
paradigm shift, transforming neuroscience into a more quantitative and 
predictive science [23]. 

The immediate impact of the Hodgkin-Huxley model was most strongly felt 
in the field of electrophysiology. By describing the ionic basis of action 
potentials, the model explained mechanistically the electrical signaling in 
neurons. This had several important implications: 

e Experimental Validation: The Hodgkin-Huxley model offered a 
framework for designing and interpreting electrophysiological 
experiments. Researchers could now test specific predictions about 
ion channel behavior and neuronal responses to electrical stimuli. 

e Standardization of Techniques: The success of the voltage clamp 
technique, which was crucial to Hodgkin and Huxley’s discoveries, 
led to its widespread adoption. This technique became a standard 
method for studying ion channels and neuronal excitability, 
facilitating the replication and extension of their findings [21]. 

e Advancement of Theory: The mathematical rigor of the Hodgkin- 
Huxley model set a new standard for theoretical work in neuroscience. 
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It demonstrated the power of combining experimental data with 
mathematical modeling to understand biological processes better [19]. 


Foundational Role in Computational Neuroscience 


The Hodgkin-Huxley model is often considered the foundation of 
computational neuroscience, which uses mathematical models and computer 
simulations to study the nervous system. The model’s equations describe how 
membrane potential changes are driven by ion channel dynamics, providing a 
basis for simulating neuronal behavior [23]. 

The Hodgkin-Huxley model enabled researchers to simulate the electrical 
activity of neurons and neural networks with unprecedented accuracy. These 
simulations have explored various aspects of neural function, including signal 
propagation, synaptic integration, and network dynamics. 

The model has been instrumental in studying neural coding, which seeks 
understanding how neural circuits represent and process information. The 
Hodgkin-Huxley model has helped researchers investigate how neurons 
encode sensory inputs and generate motor outputs by describing action 
potential generation. The principles of the Hodgkin-Huxley model have 
influenced the development of artificial neural networks, which are used in 
machine learning and artificial intelligence [28] and will be discussed in 
Chapter 3. These networks, inspired by the structure and function of biological 
neurons, have been applied to a wide range of tasks, from image recognition 
to natural language processing. 


Impact on Biomedical Research 


The Hodgkin-Huxley model has had significant implications for biomedical 
research, particularly in understanding and treating neurological disorders. By 
elucidating the ionic mechanisms underlying neuronal excitability, the model 
has provided insights into the pathophysiology of various conditions: 
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Figure 2.9: Seizures and brain activity [30] 


e Epilepsy is characterized by abnormal, excessive neuronal activity 
(Fig. 2.9 [30]). The Hodgkin-Huxley model has been used to study the 
ionic basis of epileptic seizures, helping researchers identify potential 
targets for therapeutic intervention. For example, drugs that modulate 
sodium or potassium channels can help control seizure activity [21]. 

e Chronic pain is often associated with abnormal electrical activity in 
pain pathways. The Hodgkin-Huxley model has been used to 
investigate the role of ion channels in pain perception, leading to the 
development of drugs that target these channels to alleviate pain [28]. 

The principles of the Hodgkin-Huxley model have also been applied to cardiac 
electrophysiology. The model has helped researchers understand the ionic 
basis of cardiac action potentials and arrhythmias, contributing to developing 
treatments for heart rhythm disorders. 
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Influence on Pharmacology 


The Hodgkin-Huxley model has been instrumental in pharmacology, where it 
is used to study drugs' effects on neuronal and cardiac function. The model 
provides a framework for predicting therapeutic and side effects by simulating 
drugs’ impact on ion channels. The model is used in the early stages of drug 
development to screen potential compounds for their effects on ion channel 
function. This helps identify promising candidates for further testing and 
reduces the likelihood of adverse effects. 

The Hodgkin-Huxley model provides mechanistic insights into how drugs 
interact with their targets. Researchers can optimize its efficacy and safety by 
understanding the specific ionic mechanisms a drug affects [21]. 


Educational Value 


The Hodgkin-Huxley model is widely used as an educational tool in 
neuroscience and physiology courses. Its combination of experimental data 
and mathematical rigor provides an excellent example of how biological 
phenomena can be studied quantitatively. 

The model teaches students about the electrical properties of neurons and the 
principles of electrophysiology. Its detailed, quantitative approach provides a 
clear framework for understanding complex biological processes. The 
mathematical basis of the Hodgkin-Huxley model allows for the creation of 
interactive simulations. These simulations enable students to visualize and 
manipulate the variables that influence neuronal behavior, enhancing their 
understanding of the underlying principles. 

At the end of the chapter, we will delve into some example coding examples 
to understand further the Hodgkin-Huxley model and the use of an engineering 
tool for further learning and development. 


Broader Impact, Legacy, and Continuing Relevance 


The broader impact of the Hodgkin-Huxley model extends beyond 
neuroscience and medicine. Its principles have influenced a wide range of 
fields, demonstrating the interconnectedness of scientific disciplines. The 
success of the Hodgkin-Huxley model has inspired generations of researchers 
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to pursue similar integrative approaches. Scientists can better understand 
complex biological systems by combining experimental data with 
mathematical modeling. The model’s influence extends to computer science, 
engineering, and physics. Its principles have been applied to developing new 
technologies and studying complex systems, highlighting the value of 
interdisciplinary collaboration. 

The Hodgkin-Huxley model remains relevant today as ongoing research 
continues to build on its foundations. Advances in experimental techniques and 
computational power have enabled the development of more sophisticated 
models that incorporate additional complexities: 

e New technologies, such as optogenetics and high-resolution imaging, 
provide detailed insights into neuronal function. These advancements 
complement the Hodgkin-Huxley model, allowing researchers to 
refine and extend its principles. 

e Integrating the Hodgkin-Huxley model with molecular biology and 
genetics provides a more comprehensive understanding of neuronal 
function. For example, researchers are investigating how genetic 
variations in ion channel genes contribute to individual differences in 
neuronal excitability and susceptibility to neurological diseases. 

The enduring impact of the Hodgkin-Huxley model underscores its importance 
as a foundational tool in neuroscience. Its combination of experimental rigor, 
mathematical elegance, and broad applicability continues to inspire 
discoveries and technological innovations. 


Challenges and Limitations 


Technological Advancements 


The Hodgkin-Huxley model, despite its revolutionary impact, was developed 
in an era with significant technological constraints. The tools and techniques 
available to Alan Hodgkin and Andrew Huxley were advanced for their time 
but limited compared to modern standards. Over the past several decades, 
technological advancements have addressed some of the limitations of the 
original model, enabling researchers to explore neuronal behavior in greater 
detail. 
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High-Resolution Imaging 


High-resolution imaging is one of the significant advancements that has 
enhanced our understanding of neuronal function. Techniques such as two- 
photon and electron microscopy provide detailed images of neuronal structures 
at high spatial resolution. These imaging methods allow researchers to 
visualize the distribution and density of ion channels on the neuronal 
membrane, providing insights that were not possible with the tools available 
to Hodgkin and Huxley. 

e Two-Photon Microscopy technique allows for the imaging of living 
tissue up to a millimeter deep, making it possible to study the activity 
of neurons in their natural environment. Two-photon microscopy has 
been used to observe calcium signaling in neurons, shedding light on 
the dynamics of ion channels and synaptic activity. 

e Electron microscopy provides ultra-high-resolution images of 
neuronal structures, allowing researchers to visualize the detailed 
morphology of neurons and their connections. This technique has been 
instrumental in mapping the distribution of ion channels and 
understanding their role in neuronal function. 


Computational Power 


The increase in computational power has profoundly impacted the field of 
computational neuroscience. Modern computers can perform complex 
simulations that were unimaginable in Hodgkin and Huxley's time. This 
computational power allows for modeling entire neural networks, 
incorporating thousands or even millions of neurons, each with its own set of 
Hodgkin-Huxley equations. Advances in computational power have enabled 
the simulation of large-scale neural networks, providing insights into how 
individual neurons interact to produce complex behaviors. These simulations 
can incorporate detailed models of ion channel dynamics, synaptic 
interactions, and network connectivity. 

Modern computational techniques allow for real-time simulations of neuronal 
activity, enabling researchers to study the immediate effects of changes in ion 
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channel properties or external stimuli. This capability is crucial for exploring 
dynamic processes such as learning, memory, and neural plasticity. 


Theoretical Developments 


While the original Hodgkin-Huxley model was a breakthrough, subsequent 
theoretical developments have expanded its scope and addressed some 
limitations. Researchers have developed new models incorporating additional 
complexities, providing a more comprehensive understanding of neuronal 
behavior. 

The original Hodgkin-Huxley model primarily focused on sodium (Na‘) and 
potassium (K*) channels. However, neurons possess a variety of other ion 
channels that play critical roles in their function. Subsequent models have 
incorporated these additional channels to provide a more accurate 
representation of neuronal dynamics: 

e Calcium Channels (Ca?) are essential for various neuronal processes, 
including neurotransmitter release and synaptic plasticity. Models 
incorporating calcium dynamics provide insights into the role of these 
channels in modulating neuronal excitability and signal transduction. 

e Chloride Channels (CI-) contribute to the inhibitory mechanisms in 
neurons. Including chloride dynamics in models helps explain the 
balance between excitation and inhibition in neural circuits. 

e = The original Hodgkin-Huxley model did not account for the complex, 
non-linear interactions between different types of ion channels. 
Modern models incorporate these interactions, providing a more 
realistic depiction of how ion channel activity shapes neuronal 
behavior. 

The Hodgkin-Huxley model describes the behavior of a single, isolated 
neuron. However, neurons in the brain are part of complex networks and 
constantly receive inputs from other neurons. Modern models include synaptic 
inputs to simulate how neurons integrate information from their connections. 

Including excitatory and inhibitory synaptic inputs in models helps researchers 
understand how neurons process and respond to incoming signals. This 
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integration is crucial for studying neural coding, network dynamics, and 
information processing. 

Synaptic plasticity, the ability of synapses to strengthen or weaken over time, 
is a fundamental mechanism underlying learning and memory. Models 
incorporating synaptic plasticity provide insights into how neural networks 
adapt and reorganize in response to experience. 


Alternative Models 


While the Hodgkin-Huxley model set a new standard, alternative models have 
been developed to address specific limitations and to simplify certain aspects 
for large-scale simulations. 

The Fitzhugh-Nagumo model is a simplified version of the Hodgkin-Huxley 
model. It reduces the complexity of the equations while retaining the essential 
features of excitability and action potential generation. By reducing the 
number of variables and parameters, the Fitzhugh-Nagumo model makes it 
easier to study neurons’ qualitative behavior and simulate large networks (Fig. 
2.10 [31], [32]). This model is often used in theoretical studies to explore the 
dynamics of excitable systems and to analyze phenomena such as wave 
propagation and synchronization in neural networks [20]. 

The Morris-Lecar model incorporates additional biophysical details, such as 
the role of calcium channels, to accurately describe certain types of neurons. 
The Morris-Lecar model captures the behavior of neurons that rely heavily on 
calcium for their excitability and signaling by including calcium dynamics. 
This model is beneficial for comparing the dynamics of different types of 
neurons and for studying how variations in ion channel composition affect 
neuronal behavior [28]. 


Challenges in Modeling Human Neurons 


Applying the Hodgkin-Huxley model to human neurons presents several 
challenges due to the complexity and diversity of human neural tissue. Human 
neurons exhibit a wide range of morphologies, ion channel types, and 
connectivity patterns that complicate the direct application of the original 
model. 
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Figure 2.10: Fitzthugh-Nagumo circuit (left) and Hodgkin-Huxley circuit (right) [31], [32] 


Human neurons have intricate structures with extensive branching dendrites 
and axons, which are not fully accounted for in the Hodgkin-Huxley model. 
This structural complexity influences how electrical signals are generated and 
propagated. Dendrites play a crucial role in integrating synaptic inputs. Models 
incorporating dendritic morphology and active properties accurately represent 
how neurons process information [18]. 
Axon myelination and branching affect the speed and fidelity of signal 
transmission. Including these factors in models helps explain the variability in 
conduction velocities and the mechanisms underlying action potential 
propagation. 
Different types of neurons in the human brain serve distinct functions and 
exhibit unique electrophysiological properties. For example, excitatory 
neurons, which release neurotransmitters like glutamate, and inhibitory 
neurons, which release neurotransmitters like GABA, have different ion 
channel compositions and firing patterns. 
e Neuron Types: Including the diversity of neuron types in models is 
essential for studying how different cells contribute to brain function. 
This involves characterizing each neuron type's specific ion channel 
properties and synaptic connections. 
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e Network Interactions: Understanding the interactions between 
different types of neurons in a network context is crucial for explaining 
complex behaviors such as sensory processing, motor control, and 
cognitive functions. 

Advances in experimental techniques, such as patch-clamp recordings and 
high-throughput sequencing, provide detailed data on the properties of human 
neurons. These data-driven approaches are essential for refining and adapting 
the Hodgkin-Huxley model to human neural systems. 

e Patch-clamp techniques allow for precisely measuring ionic currents 
in individual neurons. These recordings provide valuable ion channel 
kinetics and conductance data, which can be used to parameterize and 
validate models. 

e High-throughput sequencing and other molecular techniques provide 
information on ion channel expression levels and genetic variants. 
Integrating this data into models helps explain individual differences 
in neuronal behavior and susceptibility to neurological disorders. 


Recent Advances in Theoretical Neuroscience 


Ongoing research in theoretical neuroscience aims to develop more 
comprehensive models that incorporate the complexities of human neurons 
and neural networks. These models build on the foundations laid by the 
Hodgkin-Huxley model and incorporate new insights from molecular biology, 
genetics, and advanced imaging techniques. Multi-scale models integrate 
information from different levels of organization, from molecular and cellular 
levels to the whole brain. These models bridge the gap between individual 
neurons' detailed properties and large-scale neural networks' more abstract 
behavior [23]. Fig. 2.11 [33] shows a Spiking Neural Network multi-scale 
model. 

Modern models consider neurons' dynamic and non-linear behavior, such as 
voltage-gated ion channels, synaptic transmission, and plasticity. These 
dynamic systems approaches provide a deeper understanding of how neurons 
adapt and change over time in response to experience and environmental 
factors. 
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Figure 2.11: Compartmental Model for Multi Scale Models Spiking Neural Network [33] 


Future Directions and Emerging Technologies 


The field of neuroscience is rapidly evolving, with new technologies and 
theoretical developments continually pushing the boundaries of our 
understanding. Future research aims to refine and expand the Hodgkin-Huxley 


model, incorporating new data and addressing remaining challenges. 


Optogenetics allows for precise control of neuronal activity using 
light-sensitive proteins. This technology provides a powerful tool for 
testing model predictions and exploring the causal relationships 
between ion channel activity and neuronal behavior. 

Advances in high-throughput electrophysiology enable the 
simultaneous recording of electrical activity from large populations of 
neurons. This data can be used to validate and refine models, 
providing insights into the collective behavior of neural networks. 
Machine learning and artificial intelligence techniques are 
increasingly used to analyze complex neural data and develop 
predictive models of neuronal behavior. These approaches 
complement traditional modeling techniques, offering new ways to 
explore the relationships between ion channel dynamics, neuronal 
excitability, and network function. 


The continuing evolution of the Hodgkin-Huxley model underscores its 


enduring importance as a foundational tool in neuroscience. By addressing its 
challenges and limitations, researchers continually expand our understanding 
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of the intricate dynamics of neuronal function and the underlying principles of 
brain activity. 


Chapter 2: Summary 


The Hodgkin-Huxley model is a landmark in neuroscience. It offers a 
quantitative framework to understand the ionic mechanisms underlying action 
potentials. This model, developed by Alan Hodgkin and Andrew Huxley in the 
early 1950s, describes how sodium and potassium ions generate and propagate 
electrical signals in neurons, revolutionizing our understanding of neural 
function. 

The impact of the Hodgkin-Huxley model extends far beyond its initial 
application to the squid giant axon. It laid the groundwork for computational 
neuroscience, enabling the simulation of neuronal behavior and neural 
networks. The model's principles have also been instrumental in biomedical 
research, in studying neurological disorders such as epilepsy and chronic pain, 
and in pharmacology, where it helps predict drug effects on ion channels. 
Looking ahead, integrating the Hodgkin-Huxley model with emerging 
technologies and data from molecular biology promises to refine our 
understanding of neuronal dynamics further. Advances in high-resolution 
imaging, optogenetics, and high-throughput electrophysiology will enhance 
the model’s applicability and precision. Additionally, incorporating insights 
from artificial intelligence and machine learning can uncover new patterns in 
neural data, leading to more sophisticated and predictive models. 

Future innovations will address the model’s limitations, such as incorporating 
more types of ion channels, accounting for synaptic inputs, and adapting the 
model to human neurons’ complexity. These advancements will continue to 
build on the Hodgkin-Huxley model's legacy, ensuring its relevance in driving 
forward neuroscience research and applications. 
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Chapter 2: Learning activities 


Learning Activity 2.1 


Escape rooms are an effective educational tool for several 
reasons: 
e Engagement and Motivation: Escape rooms are 


inherently fun and engaging, capturing students’ 
interest and motivating them to participate actively. The immersive 
experience makes learning enjoyable, encouraging students to invest 
effort and energy in solving challenges. 

e Collaboration and Teamwork: Escape rooms require participants to 
work together to solve puzzles and achieve a common goal. This 
promotes teamwork, communication, and collaboration skills, as 
students must share ideas, listen to each other, and coordinate their 
actions. 

e Critical Thinking and Problem-Solving: The puzzles and challenges in 
escape rooms are designed to be complex and thought-provoking, 
encouraging students to use critical thinking and problem-solving 
skills. They must analyze clues, make connections, and think 
creatively to overcome obstacles. 

e Application of Knowledge: Escape rooms can be tailored to include 
subject-specific content, allowing students to apply what they have 
learned in a practical, hands-on way. This reinforces their 
understanding and helps them see the relevance of their knowledge in 
real-world scenarios. 

e Engaging Multiple Learning Styles: Escape rooms incorporate 
activities catering to different learning styles, including visual, 
auditory, and kinesthetic. This ensures that all students can engage 
with the material in a way that suits their preferred learning method. 

e Time Management and Decision-Making: The time-limited nature of 
escape rooms teaches students to manage their time effectively and 
make quick decisions. They learn to prioritize tasks, allocate 
resources, and work efficiently under pressure. 
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e Assessment and Feedback: Escape rooms can be an informal 
assessment tool, allowing educators to observe students' skills and 
behaviors. The immediate feedback from the success or failure of 
solving a puzzle provides valuable insights into students’ 
understanding and areas for improvement. 

e Emotional and Social Learning: Escape rooms' interactive and 
collaborative nature helps students develop emotional and social 
skills. They learn to cope with frustration, celebrate successes, support 
peers, and build resilience through repeated trials and errors. 

e Real-World Skills: The skills developed in escape rooms, such as 
critical thinking, teamwork, communication, and problem-solving, are 
highly valuable in real-world contexts. Students can transfer these 
skills to various academic and professional situations. 

e Customization and Versatility: Escape rooms can be customized to fit 
any educational subject or theme, making them a versatile tool for a 
wide range of topics and age groups. Educators can design puzzles and 
challenges that align with specific learning objectives and curriculum 
standards. 

We propose the following escape room based on Qualtrics and the chapter on 
Huxley-Hodgkin modeling from the book Physiology for Engineers: Applying 
Engineering Methods to Physiological Systems (Biosystems & Biorobotics, 
13), \st ed. 2016. 

Link: https://und.qualtrics.com/jfe/form/SV_dai8rHdSF9Z4S1r4 


Learning Activity 2.2 


Continuing with the buzz group educational technique, have 
groups of 3 students create a Pugh chart of the neural 


modeling tools currently available. 
A Pugh chart, also known as a decision matrix or selection 
matrix, is a tool used to compare multiple options against a set of criteria to 
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determine the best choice. Named after its creator, Stuart Pugh, this method 
helps make structured and objective decisions by evaluating alternatives based 
on specific attributes or performance indicators. 


Structure of a Pugh Chart 


1. Criteria: The vertical axis lists the criteria against which each option 
will be evaluated. 

2. Options: The horizontal axis lists the different alternatives or options 
being compared. 

3. Scoring: Each option is scored against each criterion. Scores can be 
numerical or symbolic (e.g., +, -, 0), indicating whether an option is 
better, worse, or the same compared to a baseline. 

4. Baseline: One option is typically chosen as a baseline or reference 
point against which others are compared. 


5. Summarization: The scores are summed or analyzed to determine the 
overall ranking of each option. 


Example of a Pugh Chart 


Let's create an example for evaluating neural modeling tools: 


Baseline (Tool 

Criteria ols a acolo fetches ne 
Ease of Use 0 tz - 0 
Accuracy ote 0 ar 0 
Computational Cost - at 0 0 
Documentation + - + 0 
Community Support 0 ati 0 
Total Score +1 +2 0 0 

In this example, "+" means better than the baseline, "-" means worse, and "0" 


means equal to the baseline. Tool B scores the highest overall. 
Benefits of Pugh Charts for Educational Active Learning: 
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e Structured Decision-Making: Pugh charts provide a clear and 
structured approach to comparing multiple options, teaching students 
how to evaluate choices systematically. 

e Critical Thinking: Students must define relevant criteria, analyze 
each option against these criteria, and make justified evaluations, 
promoting critical thinking and analytical skills. 

e Collaboration: Creating a Pugh chart in a group setting encourages 
discussion, debate, and consensus-building, enhancing teamwork and 
communication skills. 

e Real-World Application: Using Pugh charts simulates decision- 
making processes used in professional settings, helping students 
understand how to apply theoretical knowledge to practical 
scenarios. 

e Visual Learning: The visual representation of the comparison helps 
understand and interpret complex information, catering to visual 
learners. 

e Active Participation: Engaging students in creating and discussing 
Pugh charts ensures active participation, making the learning process 
interactive and dynamic. 

e Objective Evaluation: The method encourages objective evaluation 
based on defined criteria rather than subjective preferences, fostering 
a more impartial and fair decision-making process. 

e Overall, Pugh charts are an effective educational tool that can 
enhance active learning by promoting structured, collaborative, and 
critical evaluation of various options or concepts. 


Learning Activity 2.3 


A brainstorming activity is a creative and collaborative 
process to generate a wide range of ideas, solutions, or 
concepts related to a specific topic or problem. It begins with 


clearly defining the topic or issue to be addressed, ensuring 
that all participants understand the session's focus. Participants are then 
typically divided into small groups, although the activity can also be conducted 


88 


with the whole class, depending on its size and nature. The idea generation 
phase follows, where participants are encouraged to share their ideas freely 
and spontaneously, prioritizing quantity over quality initially. Techniques like 


' 


"yes, and..." are used to build on each other's ideas, fostering a supportive 
environment where all contributions are valued. 

A critical rule during brainstorming is to suspend judgment and criticism to 
maintain an open and encouraging atmosphere. All ideas are documented on a 
visible medium like a whiteboard, flip chart, or digital platform. After the 
initial phase, ideas are reviewed and refined, with similar ideas grouped and 
promising concepts further developed. The group then evaluates and 
prioritizes the most feasible, innovative, or relevant ideas using impact, 
feasibility, and relevance criteria. The final step involves developing action 
plans for implementing the selected ideas, assigning tasks, and setting 
timelines for execution. 

Brainstorm activities encourage creativity by promoting out-of-the-box 
thinking and exploring new ideas. They foster collaboration, enhance 
teamwork and communication among participants, and actively involve 
everyone, increasing their motivation and investment in the process. By 
drawing on the collective knowledge and experience of the group, brainstorm 
activities generate diverse perspectives and lead to innovative solutions and 
effective strategies for addressing challenges. For instance, in a session aimed 
at improving school lunches, students might be divided into small groups to 
generate and build on ideas using the "yes, and..." technique, then refine and 
prioritize the best concepts before developing an implementation plan. This 
structured yet flexible approach ensures a productive and engaging 
brainstorming experience. 

We propose the following brainstorm activity with the theme of understanding 
the basics of neuron anatomy and functionality to design a better neuron. 


@ 


rr 
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Chapter 2: Lab introduction 


In this series of lab exercises, you will explore the fundamental 
principles of neuroscience through practical applications using 
advanced engineering tools. These labs are designed to provide 
hands-on experience with MATLAB and Simulink, enabling you to 
deepen your understanding of neuron function and modeling. You 
will begin by studying the structure and communication mechanisms 
of neurons, the brain's fundamental building blocks. 

In the first lab, you will access and follow instructions to clone and 
work with a provided example locally on your machine. This 
practical task will help you become familiar with the tools and 
datasets needed for neuron analysis and visualization. By engaging 
with this hands-on example, you will build a foundational 


understanding of the methodologies used in neuroscience research. 

In the second lab, you will delve into the Izhikevich (IZH) Model, a 
mathematical model introduced by Eugene M. Izhikevich in 2003 to 
replicate various spiking and bursting behaviors observed in 
neurons. Using Simulink, you will create and simulate the IZH 
model, exploring different firing patterns of neurons. This exercise 


will provide you with practical experience in modeling complex 
neuronal behaviors and a deeper understanding of neuronal 
dynamics. 


A feed 


ad’. 
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Chapter 2: Lab Example 1 


Introduction 


The example below can be accessed either by following the 


instructions or by following them to clone it to your local 
machine. 
Prerequisites 


e Windows Subsystem for Linux (WSL) 
e Visual Studio Code (VS Code) 


e Git - 
e Python 3.10 or higher 
e Neuron 


Step-by-Step Guide 
1. Install WSL and VS Code 
e WSL: Follow the official Microsoft guide to install WSL. 
e VS Code: Download and install VS Code. 
2. Open VS Code with WSL 
1. Open VS Code. 
2. Press Ctrl+Shift+P to open the command palette. 
3. Type WSL: New Window and select it to open a new VS Code win- 
dow with a WSL session. 
3. Clone the GitHub Repository 
1. Open the terminal in VS Code (using WSL). 
Navigate to your desired directory: 


cd /path/to/your/directory 

Clone the GitHub Repository 

git clone https://github.com/your-username/your-repo. git 
Navigate to the Chapter repo you want to run examples for 
cd chapter repo 


CO Soy Gee 


Run the examples 
9. python3 example.py 
For this example, you will be using the neuronHH.py model. 
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Hodgkin-Huxley Model Example: 


Import Libraries: 


e neuron.h, gui: Imports NEURON simulation environment and GUI 
components. 

e numpy: Used for numerical calculations, especially arrays and 
mathematical functions. 

e matplotlib.pyplot: Used for plotting graphs and visualizations. 


from neuron import h, gui 
import numpy as np 
from matplotlib import pyplot as plt 


Create Sections: 


e soma and dend: Creates two sections, representing the soma and 
dendrite of a neuron. 

e dend.connect(soma(1)): Connects the dendrite to the soma at the end 
(position 1) of the soma. 


Create sections 
soma = h.Section (name='soma' ) 
dend = h.Section (name='dend') 


Topology 
dend.connect (soma (1) ) 
Define Geometry: 


e Sets the length (L) and diameter (diam) of the soma and dendrite 
sections in microns 


# Geometry 

soma.L = soma.diam = 12.6157 # microns 
dend.L = 200 # microns 

dend.diam = 1 # microns 
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Define Biophysics: 

e Sets the biophysical properties for all sections: axial resistance (Ra) 
and membrane capacitance (cm). 

e soma.insert(‘hh'): Inserts the Hodgkin-Huxley model into the soma. 

e Sets sodium, potassium, leak conductances, and the leak reversal 
potential for the soma. 

e dend.insert('pas'): Inserts a passive current mechanism into the 
dendrite. 

e Sets passive conductance and leak reversal potential for the dendrite. 


# Biophysics 
for sec in h.allsec(): 
sec.Ra = 100 # Axial resistance in Ohm * cm 


sec.cm = 1 # Membrane capacitance in micro 
Haas ay/amenla2 


soma.insert('hh') # Insert active Hodgkin-Huxley 
current in the soma 
for seg in soma: 
seg.hh.gnabar = 0.12 # Sodium conductance in 
S7em2 
seg.hh.gkbar = 0.036 # Potassium conduct-ance 
Sy emMe 


seg.-hh.gl = 0.0003 # Leak conductance in 
S/cm2 
seg.hh.el = -54.3 # Reversal potential in 


mV 


dend.insert('pas') # Insert passive current in the 
dendrite 
for seg in dend: 


seg.pas.g = 0.001 # Passive conductance in 
S/cm2 
seg.pas.e = -65 # Leak reversal poten-tial 


mV 
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Simulation: 


h.IClamp(dend(1)): Places a current clamp at the end of the dendrite. 
e delay: Delay before the stimulus starts (5 ms). 
e dur: Duration of the stimulus (1 ms). 
e amp: Amplitude of the stimulus (0.1 nA). 


# Stimulation 


stim = h.IClamp(dend(1) ) 
stim.delay = 5 

stim.dur = 1 

stim.amp = 0.1 


Recording Vectors: 


e Creates vectors to record membrane potential (v_vec) and time 
(t_vec). 

e Records the membrane potential at the middle of the soma (soma(0.5)) 
and the time from the simulation. 


# Recording vectors 

v_vec = h.Vector() # Membrane potential vector 
iG wee = n,Wecwor()) i Wilms Siceils wercicoue 

W VEC. HeCoiGl (Souna (0). 5)) s_ isis 4W)) 

i WEC. MeCome! (Ia, imSiE ic) 


Run Simulation: 
e Sets the simulation duration (simdur) to 25 ms 


e Runs the simulation 


# Run simulation 
Simdur = 25.0 
h.tstop = simdur 
lal. retolial ()} 


Plot Results: 


e Plots the membrane potential of the soma over time 
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P] 


# E 


Annotates the plot for explanation 


ot membrane potential with explanation 


.figure(figsize=(8, 4)) 


EOL (te Wee, W_Wee) 


] t.xlabel ('time (isa) 


.-ylabel ('mV') 


al 


p 
p 
p 
plt 
Pp 
ic 
h 
p 


lt.annotate('Membrane potential of soma over 
me', xy=(0.5, 0.95), xycoords='axes fraction', 
a='"center') 


1lt.show() 


User Input to Proceed Through Graphs (User prompts found in Fig 2.16): 


Prompts the user to decide whether to vary the amplitude of the cur- 
rent. 

If the user inputs "yes," varies the amplitude of the stimulus is in steps 
and plots the effect on the membrane potential of the soma. 

Prompts the user to decide whether to visualize dendrite activity. 

If the user inputs "yes," it records and plots the membrane potential of 
both the soma and the dendrite for varying stimulus amplitudes. 
Prompts the user to decide whether to test the effects of nseg (number 
of segments) on the dendritic signal. 

If the user inputs "yes," tests and plots the effects of different nseg 
values on the membrane potential of the soma and dendrite, calculat- 
ing and displaying the average error between high and low-resolution 
simulations. 

The script exits if the user decides not to proceed at any prompt. 


You must follow some prompt results as you see each graph to either process 


“yes” or not to proceed with “no” with the results. 
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® Figure 1 O 
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Figure 2.12: Membrane Potential of some over time 


Effect of varying amplitude of current on soma 
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Figure 2.13: Effect of varying amplitude of current on soma 
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Figure 2.15: Effect of nseg on dendritic signal 


Once you see the final graph (Fig 2.15), you will also see the Average percent 
error as an output in the terminal: 


97 


ts/Chapter_2 Labs/lab3_ Neuron/neuronHH. py 
Proceed to vary amplitude of current? (yes/no) 
yes 

Proceed to visualize dendrite activity? (yes/no) 


yes 
Proceed to test the effects of nseg on dendritic signal? (yes/no) 
yes 

Average error = 2.6101668670905465 


Figure 2.16: Terminal User Prompts 


To do more advanced modifications, you can always go into the code and mod- 
ify dendrite and soma equations for more advanced representations of them! 
You have learned how to create and manipulate a basic neuronal model using 
the NEURON simulation environment in this lab. Specifically, you con- 
structed a simple model consisting of soma and dendrite, defined the biophys- 
ical properties of these sections, and inserted active and passive conductances 
to simulate realistic neuronal behavior. 

You applied current stimulation to observe how the soma's membrane potential 
responds over time, then explored the effects of varying the current amplitude 
on the soma's activity. Additionally, you visualized and compared the mem- 
brane potential changes in both the soma and dendrite, demonstrating the spa- 
tial differences in signal propagation within the neuron. 

Finally, you experimented with adjusting the number of segments (nseg) in the 
dendrite to understand its impact on the accuracy of signal representation. This 
exercise highlighted the importance of model resolution in achieving accurate 
simulations of neuronal activity. 

This lab provided hands-on experience with neural modeling and an under- 
standing of how biophysical properties and model parameters influence neu- 
ronal behavior in computational simulation. 
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Chapter 2: Lab Example 2 


Overview 


This Lab Example will look at the Izhikevich (IZH) Model. 
The Izhikevich model is a mathematical model of a neuron, 
like that of Hodgkin-Huxley's, and it is designed to replicate a wide variety of 
spiking and bursting behaviors observed in neurons. Eugene M. Izhikevich 
introduced it in 2003 [34] as a more detailed model and alternative to the 
Hodgkin-Huxley model. In this lab, we will use Simulink to create the IZH 
model and then look at different firing patterns of a neuron [35]. 

The following system of ordinary differential equations describes the IZH 
Model. 

The membrane potential equation 


dv 
a 0.04v7 +5v+140-—u+l (2.5) 
The recovery variable 
du (b ) 
— =a(bv-u 
dt (2.6) 


The model also includes an after-spiking resetting condition. 


V—-C 


if v= 30m, then{, Ko uad 0.7) 


Here is a breakdown of each variable in the equation and how they work. 


Variables 


y 


e Represents the membrane potential of the neuron. 
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Measured in millivolts (mV). 

The variable indicates the neuron's voltage at any given time. 

When v reaches a threshold (e.g., 30 mV), the neuron has fired an 
action potential (spike). 


Represents the membrane recovery variable. 

It accounts for K* (potassium) activation and inactivation of Na* 
(sodium) ionic currents. 

It helps in modeling the refractory period of the neuron, during which 
the neuron is less likely to fire another spike. 

Influences the rate at which the membrane potential returns to its 
resting state after a spike. 


Represents the input current to the neuron. 

It can be thought of as the external stimulus or synaptic input the 
neuron receives. 

It influences the neuron's membrane potential and can cause it to reach 
the threshold to fire a spike. 


The time scale of the recovery variable u. 
Controls how quickly u responds to changes in v. 


Sensitivity of the recovery variable u to the membrane potential v 
subthreshold fluctuations. 
Determines how strongly u is coupled to v. 


The after-spike reset value of the membrane potential v. 
When the membrane potential v reaches the threshold (e.g., 30 mV), 
it is reset to c. 


After-spike reset increment of the recovery variable uw. 
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When the membrane potential v is reset, u is incremented by d. 
This helps to model the effect of spike-triggered adaptation, where the 
neuron's excitability is reduced after firing. 


How it works 


Membrane Potential Dynamics 


The first equation models the membrane potential dynamics v. It 
includes a quadratic term 0.04v’, a linear term 5v, and constant terms 
that drive the membrane potential. 

The term —w represents the inhibitory effect of the recovery variable 
on the membrane potential. 

The input current I can depolarize (excite) or hyperpolarize (inhibit) 
the membrane potential. 


Recovery Variable Dynamics 


The second equation describes the dynamics of the recovery variable 
u. It adjusts u based on the current value of v and the difference 
between bv and u. 

The parameter a controls the time scale, while b determines the 
sensitivity u to v. 


After-Spike Resetting 


When the membrane potential v reaches a threshold (30 mV), it is reset 
to a lower value c, simulating the neuron firing an action potential. 
Simultaneously, the recovery variable wu is incremented by d, 
increasing its value to reflect the refractory period where the neuron is 
less likely to fire again immediately. 


By adjusting the parameters a, b,c, and d, the IZH model can replicate various 


types of neuronal behavior, making it versatile and efficient for simulating 


neuronal networks. Here is a figure taken from Izhikevich et al. (2003) that 


shows some of the different types of spiking patterns that the model can 


replicate. In this lab, we will replicate a YouTube tutorial from MATLAB 


Ambassador. YouTube is an excellent resource for tutorials on using different 


engineering tools. 
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Requirements 


e = Simulink 

e MATLAB 

e YouTube 
Steps 


First, open MATLAB. Then, use this link to find the YouTube page for the 
video Simulating a Neuron by the MATLAB Ambassador. 


https://youtu.be/NnZIGC1_I0M?si=EvFK 8it5b39wfShn 


In the video, the caption includes a link to a Google Drive folder with the files 
needed to complete this lab. Here is the link as well. 


https://drive.google.com/drive/folder... 


Izhikevic Model 
nt 0.04v? + 5v+ 140 -—u+/ = f(u,v,1) 
u= a(bv — u) = h(u, v) 
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Once you have the folder open, download the NeuronParameters.m file. This 
MATLAB script will give us our parameters for the variables discussed in the 
IZH model. Once you download the file, save it and open it in your MATLAB 
as anew script. You can also double-click the file to open it in MATLAB. Once 
you open the MATLAB script, click run to ensure no errors. 

After this, go back to Google Drive and download the NeuronVideo2.slxc file. 
This is going to be the Simulink model. You can open it two ways. The first 
way is to double-click on the file and automatically open it in Simulink. The 
second way is to save the file, navigate to Simulink under the Home tab in 
MATLAB, click open under the simulation file, and then click the file. Once 
you have opened the Simulink file, it should look like Fig. 2.16. 


Simulink Model Components Description 


Here is a breakdown of the critical components of the Simulink model: 


Input Block (D): 
e Represents the input current to the neuron. 
Membrane Potential Equation Implementation: 


e Blocks calculate 0.04v”, 5v, and the constants 140 and —w. 
e The sum of these components and the input current I gives dv/dt 


Recovery Variable Equation Implementation: 
e Blocks calculate a(bv—u) to determine du/dt 


Figure 2.16: Simulink IZH model 


Function Block (Fen): 


e Handles the after-spike resetting condition. When v reaches 30 mV, it 
resets v to c and increments u by d. 


Integrator Blocks: 
e Integrate dv/dt and du/dt to update v and u over time. 
Output Blocks: 


e Display the membrane potential v and the recovery variable w. 
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Now that you have all the files and understand how they work, watch the 
YouTube video where you will model tonic spiking and bursting. Click the 
scope box to view the neurons' spiking pattern. 

Next, we will take this lab further and model more spiking patterns. We are 
going to model the inhibition-induced spiking and spiking latency. You will 
first need to return to your Neuron Parameters code in MATLAB and add the 
following parameters for the variables. 


Ss Aa1o ;o |= 
I 
fon 
(-) 


Then, you will need to change T to equal 3. This means that MATLAB will 
pull the parameters from row three of the parameter’s matrix. Click Run. Then 
go to Simulink and click Run on your model. You must click Run in MATLAB 
before rerunning the Simulink model. Double-click the Scope box to see the 
firing pattern. 

Inhibition-induced spiking, also known as post-inhibitory rebound speaking, 
is a phenomenon where a neuron generates action potentials in response to 
inhibitory input. This behavior is counterintuitive because inhibition typically 


File Tools View Simulation Help 
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Figure 2.17: Inhibition-induced spiking scope box result 
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reduces neuronal activity, yet it can trigger spiking in this case. The mechanism 
for inhibition-induced spiking involves: 

1. Inhibitory input that hyperpolarizes the neuron's membrane potential 

2. Deactivation of inward current where low threshold calcium channels 
or hyperpolarization-activated cation channels become deactivated 

3. Rebound Depolarization: When the inhibitory input is removed, the 
neuron experiences rebound depolarization. This occurs because the 
previously de-inactivated ion channel is activated, allowing an influx 
of positive ions. 

4. Action Potential Generation, where the rebound depolarization is 
strong enough, can bring the membrane potential above the threshold 
and create an action potential. 

Inhibition-induced spiking generates rhythmic activity patterns in circuits 
responsible for locomotion and respiration. Additionally, this type of firing can 
help synchronize the activity of neural networks (Chapter 3), contributing to 
the coordination of the timing of neural processing. This type of firing pattern 
shows that an inhibitory signal is just as crucial as excitatory signaling. 

Now, let’s enter the following parameters for a spiking latency. 


a 0.02 
b 0.2 
c -65 
d 

I 


6 
7 


Remember, you must change the T variable in MATLAB to the appropriate 
row in the matrix you want to pull from and click run on the MATLAB script 
before running the Simulink model. 

Spike latency in neuroscience refers to the time delay between the onset of a 
stimulus and the generation of an action potential (spike) by a neuron. This 
measure is crucial for understanding how neurons respond to external stimuli 
and process information. Spike latency plays an essential role in temporal 
encoding, as it helps neurons communicate the timing of sensory information. 
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Figure 2.18: Resonator scope box result 


For example, in sensory systems, the latency of a spike can reflect how quickly 
a stimulus is processed, with shorter latencies indicating faster responses. This 
measure provides valuable insight into the efficiency and speed of neural 
circuits, which is particularly important for tasks requiring quick reflexes or 
rapid processing. 

Additionally, spike latency affects neural coding by influencing how 
information is represented and integrated within the brain. Variations in 
latency can be used to assess changes in neural function due to various factors, 
such as diseases or experimental manipulations. Moreover, alterations in spike 
latency can signal neural plasticity and learning, indicating modifications in 
synaptic connections or neural pathways. Spike latency is a critical parameter 
for understanding the dynamics of neuronal responses and their contributions 
to sensory perception, motor control, and cognitive functions. 

That is the end of this lab example. You are encouraged to research different 
firing patterns that can be replicated using this model and try to use them in 
MATLAB and Simulink. 
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Chapter 3 


Neural Networks: The Brain’s 
Digital Doppelgangers 
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Introduction and Learning Objectives 


Neural networks are transforming our world in ways that were once thought 
impossible. They enable machines to perform tasks that previously required 
human intelligence, such as recognizing faces, translating languages, or even 
driving cars [36]. This raises several critical questions: How do neural 
networks function? What makes them so powerful? Can they genuinely mimic 
human intelligence? Understanding these concepts is essential for 
neuroengineering students, as neural networks are foundational to artificial 
intelligence (AI) and neurotechnology advancements. 

This chapter will delve into the basics of neural networks, exploring their 
structures, learning processes, and applications. We will cover the history of 
neural networks, their evolution, and their profound impact on fields like 
healthcare. By the end of this chapter, you will be able to: 


Understand the fundamental architecture and components of 
neural networks. 

Explain the process of training neural networks, including critical 
algorithms and techniques. 

Identify and describe different types of neural networks and their 


specific applications. 

Discuss the ethical considerations and potential biases in 
developing and deploying neural networks. 

Analyze real-world examples of neural network applications, 
particularly in healthcare. 


Neural networks are composed of interconnected layers of artificial neurons. 
These networks learn by adjusting the connections (weights) between neurons 
based on the data they process. This process resembles how the human brain 
adjusts synaptic strengths to learn from experiences [37]. The ability of neural 
networks to learn and adapt makes them incredibly versatile and powerful 
tools. 
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Historical Evolution of Neural Networks 


Understanding the historical evolution of neural networks provides valuable 
context for their current capabilities and future potential. The journey of neural 
networks is marked by periods of innovation, skepticism, and resurgence. 
The concept of artificial neurons dates back to the 1940s. Warren McCulloch 
and Walter Pitts proposed a mathematical model of a neuron, which could 
perform simple logical operations. Their work laid the foundation for artificial 
neural networks [38]. In 1958, Frank Rosenblatt introduced the perceptron, an 
early type of neural network that could learn to classify data into two 
categories. Despite its simplicity, the perceptron shown in Fig. 3.1 [39] 
demonstrated the potential of machine learning [40]. 

The initial excitement around neural networks was followed by periods of 
skepticism, known as AI winters. In 1969, Marvin Minsky and Seymour Papert 
published a book titled "Perceptrons," highlighting the limitations of single- 
layer perceptrons. This criticism led to declining funding and interest in neural 
network research [41]. However, the limitations of other AI approaches 
eventually became apparent, leading researchers back to neural networks. 
The resurgence of neural networks began in the 1980s with the development 
of backpropagation. Backpropagation allowed networks to learn complex, 
non-linear relationships in data. Key figures such as Geoffrey Hinton, Yann 
LeCun, and Yoshua Bengio were pivotal in advancing neural network research 
[36], [42]. 
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Figure 3.1: Perceptron Model showing the structure of the perceptron with inputs, weights, 
sum, activation function, output [39]. 
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Figure 3.2: Evolution of Neural Networks [45] 


Fast forward, 2010s saw numerous breakthroughs in neural network 


applications. Deep learning models achieved state-of-the-art performance in 


tasks such as image recognition, natural language processing, and game 


playing. Notable achievements include: 


ImageNet Competition: In 2012, a deep convolutional neural network 
developed by Hinton’s team, AlexNet, won the ImageNet Large Scale 
Visual Recognition Challenge with a significant margin, showcasing 
the power of deep learning [43]. 

AlphaGo: In 2016, Google DeepMind’s AlphaGo defeated the world 
champion Go player, demonstrating the ability of neural networks to 
master complex strategic games [44]. 

Speech and Language Processing: Neural networks have 
revolutionized speech recognition and natural language processing, 
enabling technologies like virtual assistants (e.g., Siri, Alexa) and 
machine translation (e.g., Google Translate) [36]. 
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We then transition to the age of GPTs, or Generative Pre-Trained 
Transformers, a form of Large Language Models (LLMs) that can generate 
and understand text in our natural language. This breakthrough in neural 
networks is believed to be the foundation of Artificial General Intelligence 
(AGI). ChatGPT by OpenAI has reshaped how we work and think to include 
creativity, such as the chapter images at the beginning of each chapter. This 
historical context shown in Fig. 3.2 [45] provides a foundation for 
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Figure 3.3: Natural Neuron vs. Artificial Neuron [46], [47] 
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understanding the fundamental architecture and components of neural 
networks, which we will explore in the next section. 


Fundamentals of Neural Networks 


Neural networks, inspired by the structure and function of the human brain, 
are composed of interconnected layers of artificial neurons shown in Fig. 
3.3[46], [47]. These networks can learn from data, recognize patterns, and 
make decisions, making them incredibly powerful tools in artificial 
intelligence [48]. The structure we will discuss will focus on neurons, layers, 
activation functions, and training. 


Neurons and Layers 


At the heart of a neural network are its neurons or nodes. Each neuron receives 
inputs, processes them, and produces an output. The processing involves 
computing a weighted sum of the inputs and applying an activation function 
to this sum. The neurons are organized into layers, as seen in Fig. 3.4 [49], and 
can be broadly categorized into input, hidden, and output layers [48]. 

e Input Layer: The entry point of the neural network where the raw data 
is fed into the system. Each neuron in this layer represents an input 
feature. For example, in image recognition, each neuron in the input 
layer might represent a pixel of the image. 

e Hidden Layers: One or more layers that perform most of the 
computation. These layers allow the network to learn complex 
representations of the data. Each neuron in a hidden layer takes inputs 
from all the neurons in the previous layer, applies a weight to each 
input, sums them up, and passes the result through an activation 
function. 

e Output Layer: The final layer that produces the final prediction or 
classification. The number of neurons in the output layer corresponds 
to the number of possible output classes or dimensions. For instance, 
ina binary classification problem, there would typically be one neuron 
in the output layer that outputs the probability of the input belonging 
to the positive class. 
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Figure 3.4: Basic Neural Network Architecture [49] 


Activation Functions 

Activation functions, as shown in Fig. 3.5 [50], introduce non-linearity into 

the network, enabling it to learn complex patterns that linear models cannot 

capture. Without non-linearity, the network would behave like a linear model 

regardless of its number of layers. Here are some standard activation functions: 

e Sigmoid: The sigmoid function maps any real-valued number into 

the range (0, 1), making it useful for models where the output can be 
interpreted as a probability. The function is defined as: 


Equation 1 
(3.1) 


OO) Tee 
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Hyperbolic: hyperbolic tangent function maps any real-valued number into 
the range (-1, 1). It is similar to the sigmoid function but has the advantage of 
centering the output around zero. The function is defined as: 


Equation 2 
ex—e%* 
tanh(x) = ———— v2 
anh(x) = =—— (3.2) 
ReLU (Rectified Linear Unit): The ReLU function is defined as: 
Equation 3 
f (x) = max(0, x) (3.3) 


It is computationally efficient and helps mitigate the vanishing gradient 
problem, making it the most commonly used activation function in deep 
learning [51]. The vanishing gradient problem occurs when the gradients (the 
guide on adjustments of weights due to loss) used to update neural network 
weights become very small, effectively preventing the network from learning. 


Training Neural Networks 


Training a neural network involves finding the weights that minimize the 

prediction error ona given task. This process is typically done using supervised 

learning, where the network learns from labeled examples. The critical steps 

in training a neural network are: 

1. Forward Pass: Passing the input data through the network layer by layer, 
applying weights and activation functions until the output is produced. 

2. Loss Calculation: Computing the error (loss) between the predicted output 
and the target using a loss function. 

3. Backpropagation: Calculating the gradients of the loss with respect to each 
weight using the chain rule of calculus. This step is crucial as the error is 
propagated backward through the network, as shown in Fig. 3.6 [52]. 
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Figure 3.5: Common activation Functions [50] 


4. Gradient Descent: Updating the weights using an optimization algorithm, 
typically moving the weights in the direction that reduces the loss. The 
learning rate controls the size of the weight updates. 

By iterating through these steps for many epochs (complete passes through the 

training data), the network learns to make accurate predictions. Advanced 

techniques such as batch normalization, dropout, and learning rate scheduling 
remain vital. At the same time, newer methods like Mixture-of-Experts (MoE), 
tensor parallelism, and preconditioned gradient descent are increasingly used 

to enhance training efficiency and performance and prevent overfitting [48]. 

Having established the fundamental architecture and training processes of 

neural networks, we can now explore the diverse types of neural networks and 

their specific applications in various fields. 


Types of Neural Networks 


Neural networks, inspired by the human brain, have become foundational tools 
in artificial intelligence, capable of adapting to various problems through 
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specialized architectures. Different tasks, such as image recognition or time 
series analysis, require unique configurations of neural networks to achieve 
optimal performance [36]. 

Feedforward Neural Networks (FNNs) are the simplest form of neural 
networks, where information flows unidirectionally—from input nodes 
through hidden nodes to output nodes [40]. This straightforward progression 
makes FNNs particularly effective for problems requiring linear data 
processing, such as classification tasks or essential image recognition. Their 
structure typically includes an input layer, one or more hidden layers, and an 
output layer, which makes them easy to understand and implement [48]. 
Convolutional Neural Networks (CNNs) are tailored for grid-like data 
structures like images. CNNs excel in various computer vision tasks, including 
image and video recognition, image classification, and object detection [43]. 
The core components of CNNs include convolutional layers, which apply 
filters to transform input data into feature maps by detecting spatial hierarchies 
like edges and textures. Activation functions like the Rectified Linear Unit 


(ReLU) introduce non-linearity, enabling the network to learn complex 
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Figure 3.6: Backpropagation Schematic showing the error-backpropagation used to update the 
model weights based on the gradient [53] 


patterns. Pooling layers reduce the dimensionality of feature maps, decrease 
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(a) Recurrent Neural Network (b) Feed-Forward Neural Network 


Figure 3.7: RNN and FFNN comparison [54] 


computational demands, and prevent overfitting. Fully connected layers, 
typically at the end of CNN architectures, perform high-level reasoning and 
make final classifications based on features identified by the convolutional 
layers [36]. 

Recurrent Neural Networks (RNNs) are a type of neural network designed to 
handle sequential data by allowing information to flow through cycles within 
the network. This means RNNs can remember previous inputs and use this 
memory to influence future outputs, which is especially useful for tasks like 
predicting time series data or processing natural language [37]. A schematic 
comparison of FNN and RNN is shown in Fig. 3.7 [53]. Unlike traditional 
neural networks that treat each input independently, RNNs have a hidden layer 
that retains the state of previous inputs, providing the network with historical 
context essential for making accurate predictions in tasks where the order and 
timing of data points matter [48]. 

Long Short-Term Memory (LSTM) networks found in Fig. 3.8 [54] are a 
specialized type of RNN designed to learn long-term dependencies. LSTMs 
incorporate memory cells that retain information over extended periods, 
making them particularly effective in language modeling and other sequence 
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Figure 3.8: Long-Short Term Memory Model [54] 


learning tasks [37]. The LSTM architecture includes gates regulating 
information flow: the forget gate determines what information to discard, the 
input gate decides what new information to store, and the output gate generates 
the hidden state for the next step [55]. 

Transformer networks have revolutionized natural language processing with 
self-attention mechanisms, which dynamically assign importance to different 
input parts. This capability enhances performance in language translation, text 
summarization, and question answering [56]. Transformers use positional 
encoding to maintain the order of words in a sequence and self-attention 
mechanisms to evaluate relationships within the data. Transformers consist of 
encoders, which process the input sequence into continuous representations, 
and decoders, which generate output sequences step by step [57]. 

Generative Adversarial Networks (GANs) (Fig. 3.9 [58]) consist of two neural 
networks, a generator and a discriminator, trained simultaneously. The 
generator creates data that mimics the training data, while the discriminator 
differentiates between real and generated data [48]. This adversarial process is 
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Figure 3.9: Generative Adversarial Network (GAN) Flow Diagram [58] 


samples 


Real 
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commonly used to create realistic images, videos, and other data types. As 
training progresses, the generator improves, producing increasingly realistic 
data, while the discriminator refines its ability to distinguish real from fake 
data [59]. 

With a clear understanding of the various types of neural networks, we can 
now delve into the feedback mechanisms that enhance their learning and 
performance. Feedback mechanisms, such as those found in RNNs and 
attention mechanisms, play a critical role in allowing neural networks to 
handle sequential data, focus on essential features, and improve overall 
learning efficiency. These mechanisms are pivotal in applications ranging from 
natural language processing to time series analysis, illustrating how neural 
networks continuously evolve and adapt to new challenges. Understanding 
these feedback processes will further enrich our understanding of neural 
networks’ capabilities and potential for real-world applications. 
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Figure 3.10: Recurrent Neural Network (RNN) diagram [60] 


Feedback Mechanisms in Neural Networks 


Feedback mechanisms are essential in specific neural network architectures, 
particularly in the context of RNNs and other architectures where the output 
or hidden state of the network is fed back into itself (Fig. 3.10 [60]). 

RNNs are designed to handle sequential data by maintaining a hidden state 
updated at each time step. This hidden state captures information from 
previous time steps, allowing the network to learn dependencies in the 
sequence. Training RNNs involves backpropagation through time (BPTT), 
where gradients are calculated for each time step and accumulated to update 
the weights. This process can be computationally intensive and prone to issues 
like the vanishing gradient problem [61]. 

Beyond RNNs, feedback mechanisms in Neural network architectures 
incorporate various techniques to enhance learning and performance. Attention 
mechanisms, for example, allow the network to focus on specific parts of the 
input sequence, improving tasks like machine translation and text generation 
by selectively prioritizing relevant information [62]. Residual connections in 
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deep neural networks create shortcut paths for gradients, which help mitigate 
the vanishing gradient problem and enable the training of very deep networks 
[63]. 

In Natural Language Processing (NLP) tasks, feedback mechanisms such as 
attention and recurrent structures help the network understand context and 
generate coherent text. For instance, attention mechanisms in machine 
translation focus on relevant parts of the source sentence while generating the 
target sentence, thereby improving translation quality [56]. Recurrent Neural 
Networks (RNNs) and other feedback mechanisms model temporal 
dependencies in time series analysis, making predictions based on past 
observations. This is crucial for applications like stock price prediction, 
weather forecasting, and anomaly detection in sensor data [64]. 

It is essential to understand the training processes that enable neural networks 
to learn from data to build on these varied applications of feedback 
mechanisms. This involves exploring different learning paradigms, such as 
supervised and unsupervised learning. 


Learning Paradigms 


Neural networks can be trained using different learning paradigms, with 
supervised and unsupervised learning being the most common. 


Supervised Learning 


Supervised learning, as shown in Fig. 3.11 [65], involves training a neural 
network ona labeled dataset, where each training example consists of an input 
and a corresponding target output. The network learns to map inputs to outputs 
by adjusting its weights to minimize the prediction error. This iterative learning 
process requires substantial labeled data to achieve high accuracy [48]. 

Collecting and preparing data is crucial in supervised learning. The dataset 
must include many labeled examples that are accurately annotated to train the 
network effectively. The quality and quantity of the data directly impact the 
network's performance [66]. During training, the network processes input data 
through layers, applies activation functions, and generates predictions. The 
error between the predicted and actual outputs is computed using a loss 
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function. This error is then propagated backward through the network 
(backpropagation), and the weights are updated using an optimization 
algorithm like gradient descent to reduce the error [42]. 

To ensure the model generalizes well to new data, a separate validation set 
(typically 80% training: 20% test) is used during training to monitor 
performance and prevent overfitting. After training, the model is tested to 
assess its accuracy, generalizability, and production readiness [67]. Supervised 
learning is widely used in various applications, such as image classification, 
speech recognition, and natural language processing. For instance, in image 
classification, the network is trained on labeled images and learns to predict 
the category of new images. In speech recognition, the network converts 
spoken language into text by learning from audio recordings and their 
transcriptions [68]. 
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Figure 3.11: Supervised Learning Process Flow [65] 
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Unsupervised Learning 


In unsupervised learning, the network learns from unlabeled data, where the 
goal is to identify patterns and structures without explicit target outputs. This 
paradigm is useful for exploratory data analysis and discovering hidden 
structures in data [69]. 

Clustering algorithms group similar data points based on their features. Neural 
networks, such as autoencoders and deep clustering models, can learn to 
represent data in a lower-dimensional space and identify clusters. This 
technique is useful in applications like market segmentation and biological 
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Figure 3.12: Comparison of Supervised and Unsupervised Learning [71] 
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data analysis, where the objective is to group similar entities [70]. A 
comparison of supervised and unsupervised learning can be seen in Fig. 3.12 
[71]. 

Techniques like Principal Component Analysis (PCA) and t-Distributed 
Stochastic Neighbor Embedding (t-SNE) reduces the number of variables 
under consideration while preserving the essential structure of the data. Neural 
networks like autoencoders can also minimize dimensionality by learning 
compact data representations. This is beneficial for data visualization and 
noise reduction [72]. 

Unsupervised learning is also used to detect anomalies or outliers in data. For 
example, neural networks can analyze traffic in network security to identify 
unusual patterns indicating cyber-attacks. In manufacturing, anomaly 
detection helps identify product defects by recognizing deviations from normal 
patterns [73]. 


Semi-Supervised Learning 


Semi-supervised learning combines a small amount of labeled data with a large 
amount of unlabeled data during training. This approach is useful when 
labeling data is expensive or time-consuming. Neural networks can leverage 
unlabeled data to learn underlying structures and improve performance on the 
labeled data [74]. 

The network is initially trained on the labeled data and then uses the learned 
features to predict the unlabeled data. These predictions are refined iteratively, 
improving the model's accuracy. Semi-supervised learning bridges the gap 
between supervised and unsupervised learning, making efficient use of both 
labeled and unlabeled data [75]. Semi-supervised learning is used in various 
fields, such as text classification, image recognition, and bioinformatics. For 
example, in text classification, a small set of labeled documents can be used to 
train the network, predicting the categories of a larger set of unlabeled 
documents. 


Reinforcement Learning 


Reinforcement learning involves training an agent to make decisions by acting 
in an environment to maximize cumulative rewards. The agent learns from 
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feedback through rewards or penalties and adjusts its actions based on this 
feedback [76]. 

The agent interacts with the environment, observes the outcomes of its actions, 
and receives rewards or penalties. It uses this information to update its policy, 
a strategy for choosing actions. Over time, the agent learns to maximize its 
cumulative rewards by refining its policy [77]. Reinforcement learning is used 
in robotics, game-playing, and autonomous systems. For example, in robotics, 
an agent learns to perform tasks through trial and error, such as navigating a 
maze or manipulating objects. In-game playing, reinforcement learning 
algorithms, such as those used in AlphaGo, have achieved superhuman 
performance in complex games by learning optimal strategies through 
extensive self-play [44]. 

Having explored the various learning paradigms, it is essential to understand 
how neural networks can mimic certain aspects of human intelligence and 
recognize complex patterns, which will be covered in the following sections. 


Mimicking Human Intelligence 

Neural networks can mimic certain aspects of human intelligence, such as 
pattern recognition, language understanding, and decision-making. However, 
they do so in a fundamentally different way from the human brain. Neural 
networks operate based on mathematical computations and statistical learning, 
whereas human intelligence involves consciousness, emotions, and 
understanding [78]. A comparison can be found in the table below. 

Neural networks are designed to solve specific tasks by learning from data. 
They excel at tasks that require pattern recognition and prediction, such as 
image and speech recognition, language translation, and game playing. 
However, neural networks lack the general intelligence and adaptability of the 
human brain. They are limited to the tasks they are trained on and do not 
possess the ability to understand or reason beyond their training. This 
distinction highlights the difference between narrow AI, where neural 
networks excel, and Artificial General Intelligence (AGI), which remains an 
unsolved challenge in the field [61]. For further understanding, table 2.1 [79] 
compares ANN and Natural Neural Networks, our brain. Now that you have a 
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sense of function, architecture, and the types of Neural Networks, let’s explore 
some real-world examples to strengthen your understanding of the topic. 


Table 2.1: Artificial Neural Networks vs Biological Neural Networks [76] 


— Artificial Neural Biological Neural 
Characteristics 
Network (ANN) Network (BNN) 
Faster in processing Slower in processing 
Speed information. Response information. The response 
time is in nanoseconds. time is in milliseconds. 
Massivel lel 
Processing Serial processing. ee eee 
processing. 
A highly complex and 
: : d twork of 
: Less size & complexity. It | . ee ee 
Size & interconnected neurons 
‘ does not perform complex _ 
Complexity re een ene containing neurons of the 
’ e | order of 10'! with 10% 
interconnections. 
Varies based on design, : 
Number of ; : nee Approximately 10"! (100 
typically in the range of _ 
Neurons me billion) neurons. 
thousands to millions. 
Information storage is 
: ; adaptable, adding new 
Information storage is : : ee 
‘ information by adjusting 
replaceable, meaning . . 
Storage : : the interconnection 
replacing new data with . 
strengths without 
old data. : 
destroying old 
information. 
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Fault Tolerance 


Fault intolerant. Corrupt 
information cannot be 
retrieved in case of 
system failure. 


Fault-tolerant. 
Information storage is 
adaptable, adding new 
information by adjusting 
the interconnection 
strengths without 
destroying old 
information. 


Control 
Mechanism 


There is a control unit for 
controlling computing 
activities. 


There is no specific 
control mechanism 
external to the computing 
task. 


Real-World Example Solving Real-World 


Problems 


One of the most profound applications of neural networks is healthcare, 
specifically in diagnosing diseases through medical imaging, as shown in Fig. 
3.13 [80]. The traditional process of diagnosing diseases like cancer from 
medical images such as mammograms or CT scans is time-consuming and 
requires highly skilled radiologists. Even then, human analysis is subject to 
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Figure 3.13: Mammogram Image [80] 
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(c) Method Result 


variability and error. This is where neural networks, particularly deep learning 
models, have shown transformative potential [81]. 

Breast cancer is one of the leading causes of cancer-related deaths among 
women worldwide. Early detection significantly improves the chances of 
successful treatment and survival. However, interpreting mammograms to 
detect early signs of cancer is challenging due to the subtle differences between 
benign and malignant tissues [82]. 

CNNs are designed to automatically and adaptively learn spatial hierarchies of 
features from input images. Large datasets of mammogram images are 
collected and annotated by expert radiologists. The CNN model is trained 
using these labeled images to recognize patterns and features that differentiate 
normal tissues from cancerous ones. Once trained, CNNs can analyze new, 
unseen mammogram images to predict the likelihood of cancer, assisting 
radiologists in making more accurate and faster diagnoses [83]. Fig. 3.14 [84] 
shows the inputs, channels, activation, and convolutional neural network that 
allow for this type of detection of patterns and learning. 

In addition to breast cancer detection, neural networks have significantly 
impacted the diagnosis of brain disorders through advanced imaging 
techniques. Early detection and accurate diagnosis are crucial in treating brain 
disorders such as Alzheimer's disease, brain tumors, and stroke. The ability of 
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Figure 3.14: Convolutional Neural Network Architecture [84] 
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neural networks to analyze and interpret complex brain images offers 
substantial improvements over traditional methods, which are often time- 
consuming and prone to variability. 

Convolutional Neural Networks (CNNs) have been extensively used in brain 
imaging due to their proficiency in processing grid-like data structures such as 
MRI scans, as shown in Fig. 3.15 [85]. A prominent application of CNNs in 
brain imaging is the early detection of Alzheimer's disease. Alzheimer's 
disease is a progressive neurodegenerative disorder characterized by the 
accumulation of amyloid plaques and tau tangles in the brain, leading to 
cognitive decline and memory loss. Early detection is critical as it can 
significantly improve treatment effectiveness and slow the disease's 
progression. In the early detection of Alzheimer's disease, large datasets of 
MRI scans are collected from patients diagnosed with the disease and healthy 
controls. Medical experts annotate these images to identify brain regions 
affected by Alzheimer's. The CNN model is then trained using these labeled 
images to learn the distinguishing features of Alzheimer's disease. The 
network's layers automatically extract relevant features from the input images, 
such as differences in brain volume and the presence of amyloid plaques. 
During training, CNN learns to recognize patterns in MRI scans indicative of 
Alzheimer's disease. The network adjusts its weights through backpropagation 
and gradient descent to minimize the prediction error. This iterative process 
involves multiple epochs, where the network repeatedly processes the training 
data and refines its predictions. Data augmentation, regularization, and 


a b c 
Figure 3.15: MRI images of (a) Mild Demented, (b) Moderate Demented, (c) Non-Demented, 
and (d) Very Mild Demented. Though the images may seem meaningless, a CNN identified the 
correct Alzheimer’s stage with a >95% accuracy [85] 
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dropout enhance the training process and prevent overfitting, ensuring the 
model generalizes well to new, unseen data. 

Once trained, CNNs can analyze new MRI scans to predict the likelihood of 
Alzheimer's disease. The network's predictions assist radiologists in making 
more accurate and timely diagnoses. Studies have shown that CNNs can 
achieve high sensitivity and specificity in detecting Alzheimer's, often 
outperforming traditional diagnostic methods. For instance, a study by Abdul 
Azeem et al. (2021) demonstrated that a CNN-based model could classify 
Alzheimer's disease with an accuracy of over 97%, highlighting the potential 
of deep learning in clinical practice. 

Recent advancements in neural network techniques have significantly 
improved the early diagnosis and classification of Alzheimer's disease. For 
example, a hybrid deep learning approach combining BiLSTM and ANN 
models has shown promise in early diagnosis by capturing long-range 
dependencies and temporal dynamics within medical records [86]. 
Additionally, a novel multi-scale attention-based pseudo-3D convolutional 
neural network has been developed for Alzheimer's diagnosis using structural 
MRI data. This method leverages attention mechanisms to focus on critical 
features, enhancing the network's ability to classify different stages of 
Alzheimer's disease accurately. 

Another innovative approach involves using a deeply supervised adaptable 
neural network that integrates multitask feature extraction for diagnosing and 
classifying the severity of Alzheimer's disease. This method utilizes traditional 
machine learning techniques and deep learning models to analyze MRI 
images, comprehensively assessing Alzheimer's severity [87]. 

These applications of neural networks in brain imaging underscore their 
transformative potential in healthcare. By leveraging advanced machine 
learning techniques, neural networks provide potent tools for early detection, 
accurate diagnosis, and effective treatment planning for brain disorders. As 
research progresses, integrating neural networks with medical imaging is 
expected to yield even more sophisticated diagnostic capabilities, ultimately 
enhancing patient care and outcomes. 
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Ethical, Privacy, and Security 


Considerations in Neural Network Development 


As neural networks become more prevalent, ethical, privacy, and security 
considerations in their development and deployment are increasingly 
important. Neuroengineering students must be aware of these issues to ensure 
the responsible use of this technology. 

One significant concern is bias and fairness. Neural networks can inadvertently 
learn and perpetuate biases in the training data, leading to unfair or 
discriminatory outcomes. For example, a facial recognition system trained on 
a dataset with limited diversity may perform poorly on specific demographic 
groups. Addressing bias and ensuring fairness involves collecting diverse and 
representative datasets, developing techniques to detect and mitigate bias, and 
providing transparency and accountability in model development [88]. 

Think about a wearable that was trained to look at skin cancer. The skin 
training data set may be primarily Caucasians due to the location of the clinical 
trials and the focus on diversity. This poses a significant setback in the ability 
of the device to detect skin cancer in those with darker pigments, rendering the 
device ineffective in many regions of the world. Privacy is another critical 
issue. 

Neural networks often require vast amounts of data to achieve high 
performance, including sensitive personal information such as medical 
records, financial transactions, and social media activity. Ensuring data privacy 
is crucial to protect individuals’ rights and maintain public trust. Data 
anonymization, differential privacy, and federated learning are being 
developed to enable neural networks to learn from data without compromising 
individual privacy [89], [90]. 

In addition to privacy, security is a significant concern. Neural networks are 
vulnerable to adversarial attacks, where malicious inputs are crafted to deceive 
the network into making incorrect predictions. These attacks can have serious 
consequences, especially in critical applications such as autonomous vehicles, 
healthcare, and security systems. Developing robust neural networks that can 
withstand adversarial attacks is an active area of research [91]. 
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Ethical AI guidelines play a crucial role in addressing these concerns. 
Developing and adhering to such guidelines is essential for responsible neural 
network development. Like that of Fig. 3.16 [92], ethical frameworks should 
emphasize fairness, transparency, accountability, and privacy. Compliance 
with relevant laws and regulations and engaging with diverse stakeholders are 
crucial to ensure ethical AI development [93]. Addressing the ethical 
considerations in neural network development involves mitigating biases, 
protecting privacy, providing security, and adhering to ethical guidelines to 
foster transparency, accountability, and fairness in AI systems. These all 
contribute to what the future of AI and Neural Networks will look like. 
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Figure 3.16: Ethical AI Framework from [92] 
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Future Directions and Challenges 


The field of neural networks is rapidly evolving, with ongoing research 
addressing current challenges and exploring new frontiers. 

Researchers continually develop new neural network architectures to improve 
performance and expand capabilities. Transformer networks, for example, 
have revolutionized natural language processing with their self-attention 
mechanisms [56]. Graph neural networks (GNNs) are designed to process 
graph-structured data, such as social networks and molecular structures [94]. 
Spiking neural networks (SNNs) aim to mimic biological neurons' 
asynchronous and event-driven nature, potentially leading to more efficient 
and biologically plausible models [95]. 


Interdisciplinary Research 


The intersection of neural networks with other fields, such as neuroscience, 
biology, and physics, offers exciting opportunities for interdisciplinary 
research. Collaborations between AI researchers and neuroscientists are 
leading to a deeper understanding of brain function and the development of 
biologically inspired neural networks. In biology and medicine, neural 
networks are used to analyze biological data, model biological processes, and 
develop personalized medicine approaches. Quantum neural networks (QNNs) 
are an emerging field exploring the potential of quantum computing to perform 
computations that are infeasible for classical computers [96]. 


Ethical and Societal Impact 


As neural networks become more integrated into society, addressing their 
ethical and societal impact remains a critical challenge. Ensuring ethical AI 
development involves ongoing research into bias mitigation, fairness, and 
transparency. Understanding the societal implications of AI, such as its impact 
on employment, privacy, and security, is essential for responsible AI 
deployment. Developing policies and governance frameworks to regulate AI 
development and use is crucial for ensuring that AI benefits society as a whole 
[93]. 
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Recent Advances and Future Directions 


1. 


Neural Architecture Search (NAS): Automated methods for 
designing neural network architectures are gaining traction. NAS 
techniques aim to identify optimal architectures for specific tasks, 
potentially outperforming human-designed models [97]. 

Federated Learning: This approach trains models across 
decentralized devices while preserving data privacy. Federated 
learning is particularly relevant for applications involving sensitive 
data, such as healthcare [98]. 

Explainable AI (XAJI): As AI systems become more complex, it is 
crucial to ensure their transparency and interpretability. XAI research 
focuses on developing methods to make neural network decisions 
understandable to humans [99]. 

AJ and Climate Change: Neural networks model climate systems and 
predict environmental changes. This research is vital for developing 
strategies to mitigate the impact of climate change [100]. 

Advanced Reinforcement Learning: Techniques such as meta- 
reinforcement learning and hierarchical reinforcement learning are 
expanding the capabilities of AI systems to learn complex tasks 
efficiently [101]. 

Neuromorphic Computing: Inspired by the human _ brain, 
neuromorphic computing aims to develop hardware that mimics 
neural architectures, potentially leading to more efficient and powerful 
AI systems [102]. 


As we conclude this chapter, we've explored the transformative power of 


neural networks and their profound impact on various fields. From their 


historical evolution and fundamental workings to the diverse types and real- 


world applications, neural networks have demonstrated their capability to 


solve complex problems. Ethical, privacy, and security considerations are 


paramount in their development, ensuring responsible and fair use. The future 


holds exciting advancements in architecture and interdisciplinary research, 


promising to enhance further the capabilities and applications of neural 


networks, ultimately benefiting society as a whole. As you move forward, 
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consider these key aspects and their implications in the broader context of 
artificial intelligence and neurotechnology. 


Chapter 3: Summary 


In this chapter, we have systematically examined the core principles and prac- 
tical applications of neural networks. We began by detailing the fundamental 
architecture of neural networks, including the roles of neurons, layers, and ac- 
tivation functions, which collectively enable these systems to process and learn 
from data. This foundation was further contextualized by tracing the historical 
development of neural networks, highlighting key innovations that have driven 
their current capabilities, particularly in deep learning. 


We then explored various types of neural networks, each designed to address 
specific computational tasks. Convolutional Neural Networks (CNNs) were 
shown to excel in image processing, while Recurrent Neural Networks (RNNs) 
demonstrated their utility in handling sequential data. These architectures have 
been instrumental in advancing fields such as healthcare, where neural net- 
works are now critical in tasks like disease diagnosis and treatment planning. 


Ethical considerations were also addressed, emphasizing the importance of 
mitigating bias, safeguarding privacy, and enhancing security in developing 
and deploying neural networks. As these technologies become more integrated 
into critical applications, it is essential to create robust frameworks to ensure 
responsible use. 


Looking forward, the field of neural networks is poised for continued advance- 
ment. Ongoing research into new architectures and interdisciplinary ap- 
proaches will likely yield further performance improvements and expand these 
systems' applicability. However, with these developments comes the responsi- 
bility to manage the societal impacts of neural networks, ensuring that their 
deployment is ethical and beneficial to various stakeholders. 


In the coming chapters, we will focus on neuroengineering technologies such 
as EEG, fMRI, and Brain-Computer Interfaces (BCI). These tools are essential 
for understanding and treating neurological diseases and offer new avenues for 
interfacing directly with the human brain. As we delve into these technologies 
and their applications, we will continue to build on the foundation laid by our 
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exploration of neural networks, connecting these advances to the broader field 
of neurotechnology. 


This chapter has provided a comprehensive overview of neural networks, 
grounding your understanding in their theoretical underpinnings and practical 
applications. As you move deeper into the study of artificial intelligence and 
neurotechnology, the knowledge gained here will serve as a critical foundation 
for understanding and developing future innovations in these fields. 
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Chapter 3: Learning activities 


Learning Activity 3.1 


Using the Buzz groups technique, propose the following 
challenge as the students to give an example of how you are 
using NN today. Also, this classroom assumes the student is 
working on a semester-long engineering project; how would 
you use them in your project? 


Learning Activity 3.2 


A quescussion, a blend of "question" and "discussion," is an 
interactive and participatory teaching and _ learning 
technique where participants engage in a dialogue of 


questions. This method stimulates critical thinking, active 
participation, and deeper topic exploration. The rules of a quescussion are 
simple: participants can only contribute by asking questions, not by making 
statements or providing answers. 

Key Features of a Quescussion: 

e Question-Only Dialogue: Participants can only speak in the form of 
questions, avoiding statements, answers, or declarative comments. 

e Open-Ended Questions: Questions are typically open-ended to 
promote thought, reflection, and further inquiry. 

e Group Participation: All participants are encouraged to contribute 
questions, fostering an inclusive and engaging environment. 

e Facilitation: A facilitator may guide the process, ensuring the 
discussion remains focused and productive while maintaining the 
question-only rule. 

e Reflective Learning: The format encourages participants to think 
deeply and consider multiple perspectives, enhancing critical 
thinking and understanding. 
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Example: Here’s an example of a quescussion on the topic of "Artificial 
Intelligence in Education": 

Facilitator: "Let's begin our quescussion on the impact of artificial intelligence 
(AI) in education. Remember, you can only ask questions; no statements or 
answers are allowed. Who wants to start?" 

Participant 1: "How is AI currently being used in classrooms?" 

Participant 2: "What are the potential benefits of AI for personalized learning?" 
Participant 3: "Could AI help teachers with administrative tasks?" 

Participant 4: "What are the risks associated with relying on AI for grading and 
assessments?" 

Participant 5: "How might AI affect the role of teachers in the future?" 
Participant 6: "Are there ethical considerations we must address when 
implementing AI in education?" 

Participant 7: "What are some examples of successful AI tools already used in 
education?" 

Participant 8: "How can we ensure AI is accessible to all students, regardless 
of their socio-economic background?" 

Participant 9: "What training do teachers need to use AI effectively?" 
Participant 10: "How can AI help identify and support students with learning 
difficulties?" 

Participant 11: "What are the long-term impacts of AI on student privacy?" 
Participant 12: "How do students feel about interacting with AI in their 
learning environments?" 

Facilitator: "Great set of questions! Let's keep going. Who has the next 
question?" 

Participant 13: "What kind of data does AI need to personalize learning 
experiences?" 

Participant 14: "Can AI adapt to different learning styles and paces?" 
Participant 15: "How do we measure the effectiveness of AI in education?" 
This format encourages continuous inquiry and exploration of the topic, 
allowing participants to dive deeper into the subject matter through a series of 
interconnected questions. 
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Following this technique, propose the following quescussion; start with the 
bold statement: “Unsupervised learning is better than supervised learning.” 


Learning Activity 3.3 


The following website provides an easy way to demonstrate 
how machine learning works. By following these simple 
steps, students can learn and apply the concepts: 


1. Collect examples of the items you want the system 
to recognize. 
Train a computer using these examples to recognize the items. 
3. Create a game in Scratch that leverages the computer's recognition 
capabilities. 
Have your students apply this procedure to pattern recognition in a 
neuroengineering-related process, allowing them to select which aspect they 
want to classify. 


Game: https://machinelearningforkids.co.uk/ 


@ 


rt 
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Chapter 3: Lab introduction 


In this series of lab exercises, you will introduce the world of neural 
networks and their diverse applications, ranging from general 
machine learning tasks to specialized neuroengineering projects. 
These labs will provide practical experience in developing, training, 
and evaluating neural networks, enhancing your understanding of 
their functionality and potential applications. 

You will start by exploring the fundamental concepts of neural 
networks, which are inspired by the structure and functioning of the 
human brain. These powerful tools can learn from data and make 
intelligent decisions, proving essential in fields such as image 


recognition, natural language processing, and neuroengineering. 


Through hands-on activities, you will gain insights into how neural 
networks are structured and how they operate. 

Next, using a fictional dataset, you will apply your knowledge by 
developing a neural network to predict injury risk from vehicle crash 
data. This exercise will involve generating sample data, building and 
training the neural network, and testing its performance with input 
data to evaluate its functionality. By the end of these labs, you will 
have practical experience applying neural networks to research 
datasets, preparing you for further exploration in machine learning 
and neuroengineering. 


\ feed 
\' 


141 


Chapter 3: Lab Example 1 


Introduction 


This practical lab example will explore the fascinating 


world of neural networks and their applications in general 
machine learning and specialized neuroengineering tasks. Neural networks, 
inspired by the structure and functioning of the human brain, are powerful 
tools capable of learning from data and making intelligent decisions. They 
have become essential in various fields, from image recognition and natural 
language processing to more advanced applications like neuroengineering. 
First, we will start with a simple yet classic example of using neural networks: 
training a model to recognize handwritten digits from the MNIST dataset. This 
example introduces fundamental concepts and provides hands-on experience 
in building, training, and evaluating a neural network using TensorFlow and 
Keras. The MNIST dataset, comprising 70,000 images of handwritten digits, 
is a standard benchmark for evaluating image classification algorithms. 
Following this introductory example, we will delve into a more advanced and 
specialized application in the field of neuroengineering: training a neural 
network to recognize artifacts in MRI images. MRI artifacts can obscure the 
quality of scans and hinder accurate diagnosis and treatment planning. By 
training a neural network to identify and correct these artifacts, we can enhance 
the precision of neuroimaging techniques, ultimately improving patient 
outcomes. 

This lab will guide you through downloading the necessary tools, 
preprocessing data, building neural network models, and training and 
evaluating these models on specific datasets. Whether new to neural networks 
or looking to expand your knowledge into neuroengineering applications, 
these examples will provide a solid foundation and practical skills to apply in 
future projects. 


Example 1: MNIST Handwriting Training 


To avoid any library or Python environment dependencies, we will use Google 
Collaboratory. Type it into Google and sign up for free with a Gmail account. 
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Like with a Jupiter Notebook, you will be well on your way to training and 
testing a neural network. 
Step 1: Install Required Libraries 


First, you must install TensorFlow (if you are using your local machine or if 
Collab errors arise). You can do this using pip: 


pip install tensorflow 


Step 2: Import Libraries 


Next, we need to import the necessary libraries in our Python script: 
Import tensorflow as tf 

from tensorflow.keras.datasets import mnist 

from tensorflow.keras.models import Sequential 
from tensorflow.keras.layers import Dense, Flatten 
from tensorflow.keras.utils import to categorical 


Step 3: Load and Preprocess the Data 


We will load the MNIST dataset and preprocess it for our neural network: 


# Load the MNIST dataset 
(Gs ‘eigen, 37 icieelim)), (G2 teste, W wesc) = 
mnist.load_ data() 


# Normalize the input data 
Mvikain =< Eraimeastypet tloats2 yf 255 
x test = x test.astype('float32') / 255 


# Convert labels to categorical one-hot encod-ing 
y_ train = to categorical(y train, 10) 
y_ test = to_categorical(y test, 10) 


Step 4: Build the Neural Network Model 


We will build a simple neural network model: 
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i? limalicigllawe clas imercleil 
model = Sequential () 


# Add layers to the model 
mode ladd(Plarcen(inpur wshape=(26, 26))) 97 Plarcen 
the input shape (28x28 pixels) 

model.add(Dense(128, activation='relu')) # Adda 
fully connected layer with 128 units and ReLU 
activation 
model.add(Dense(10, activation='softmax')) # Adda 
fully connected layer with 10 units (one for each 
class) and softmax activation 


# Compile the model 
model.compile(optimizer='adam', 
IGSSS" CaicSecioicalesill Ceossemiciccjoy! , wee 
rics=['accuracy']) 


Step 5: Train the Model 


We will train the model using the training data: 


# Train the model 
MOGeIN tile) (KWEGakeny waver ral, epochs — 5), 
batch _size=32, validation_split=0. 2) 


Step 6: Evaluate the Model 


Finally, we will evaluate the model using the test data: 


# Evaluate the model 

test_loss, test accuracy = mod-el.evaluate(x test, 
WV icSSic)) 

jeiaine(Ge Wes losses (eesic lose) ’)) 

print (Mes eEdCcuracy: mpeCSicma eC Ulsd eis) as) 
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Downloading data from https: //storage.googleapis.com/tensorflow/tf-keras-datasets/mnist .npz 


11490434/11490434 [==============================] - 0s Ous/step 

Epoch 1/5 

1500/1500 [ ass = sueceesessesses | 8s Sms/step - loss: 6.2919 - accuracy: 0.9160 - val_loss: @.1622 - val_accuracy: 0.9539 
Epoch 2/5 

1590/1500 [ ] - 5s 3ms/step - loss: @.1295 - accuracy: @.9621 - val_loss: @.1149 - val_accuracy: 0.9642 
Epoch 3/5 

1500/1500 [==============================] - 7s 5ms/step - loss: 0.0878 - accuracy: 0.9737 - val_loss: @.1011 - val_accuracy: 0.9693 
Epoch 4/5 

1500/1500 [ ] - 5s 3ms/step - loss: 06,0648 - accuracy: @.9799 - val_loss: @,0963 - val_accuracy: 0.9709 
Epoch 5/5 

1500/1500 [== ===] - 7s Sms/step - loss: @.0500 - accuracy: @.985@ - val_loss: 0.0886 - val_accuracy: 0.9732 


313/31 
Test loss 
Test accur 


1s 2ms/step loss: 0.0865 accuracy: 0.9748 


Figure 3.17: Evaluating the model 


It doesn’t look fancy, but you have trained and tested a Neural Network (Fig. 
3.17). 

Let’s do something a little more challenging to continue on this track. Let’s 
train a model to recognize certain artifacts or features in brain MRI images 
with MRI Visualization.py. 

NOTE: This code should be used in Google Colab (free) or Jupyter note- 
book by placing the code in code cells within the notebook. 

We will start with loading the data, processing it, Training, & Testing, and then 
we can start with some ML Predictions based on the inputted data. 


# Define the path to the participant data file in your Google 
Drive 

participant_data_path = ‘'/content/drive/MyDrive/Brain-MRI- 
Age-Classification-using-Deep-Learning/ds@00228-1.1.0-sub- 
set/derivatives/participants.tsv' 


# Load participant data 

participant_data = pd.read_csv(participant_data_path, 
sep='\t') 

participant_data[ 'AgeClass' ] = pd.cut(partici- 
pant_data[‘'Age'], bins=[3, 6, 13, np.inf], labels=[ 'Ages3- 
5', 'Ages7-12', 'Adults']) 


# Display the first few rows of the participant data 
participant_data.head() 


145 


(4) 


Tom 


TOM Booklet 


WPPSI WPPSI pecs Scanlog: 


participant_id Age AgeGroup child Adult Gender Handedness Booklet- 20 ¢neq. FB_Composite FBGroup BD 8D KBIT_raw KBIT_standard oetary scanner 
matched raw scaled 
NOFB 
0 —sub-pixar001 4,774812 4yo child M R 0.80 0.736842 60 pass 220 130 NaN NaN 3.0 371 
1 sub-pixar002 4.856947 4yo child F R 0.72 0.736842 40 inc 180 90 NaN NaN 20 3m 
2 — sub-pixar003 4,153320 4yo child F R 0.44 0.421053 3.0 inc 150 90 NaN NaN 3.0 am 
3 subpixar004 4.473648 4yo child F R 0.64 0.736842 20 fal 17.0 100 NaN NaN 30 3am 
4 —sub-pixarO05 4.837782 Ayo child F R 0.60 0.578947 40 inc 130 5.0 NaN NaN 20 an 


Figure 3.18: Data Structure 


The structure of the data is shown in Fig. 3.18; we won't share images of the 
epochs and training, but in Fig. 3.19, you will see the output of a convolution 
matrix, predictions, and Occlusion Maps of the images of the brain. 


Original Image (Label: Ages3-5) Occlusion Sensitivity Map 
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Figure 3.19: Output of convolution matrix, predictions and Occlusion maps 
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Chapter 3: Lab Example 2 


Introduction 


In this lab example, we will develop our neural network 
(NN) to predict injury risk from vehicle crash data using 
artificial data to illustrate how you could apply NNs to your research. We will 
generate sample data, build and train the NN, and then test it with input data 
to evaluate its functionality. To reiterate, this is an entirely fictional dataset 
meant to illustrate the application of NNs to various research datasets. 


Exercise Instructions 


Step 1: Generate Sample Data 

First, we will generate synthetic data to simulate vehicle crash database 
information. Imagine this data could be any form of information, whether 
training for images as shown in Fig. 3.20 or a large dataset with a range of 


Scatter Plot of Delta-V vs. Peak Acceleration 
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Figure 3.20: Delta-V vs. Peak Acceleration Scatter Plot 
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variables as in this example. Obviously, the type of network you design will 
change, but this is meant to be a very basic exercise. We will include vehicle 
type, crash severity, peak acceleration, and corresponding injury risk for this 
artificially generated data. See the GitHub for the complete code. For now, our 
generated data is artificially skewed with some variability and noise to 
introduce realistic trends in data to improve this theoretical model’s prediction 
accuracy (increased severity leads to increased injury risk) while attempting to 
include some degree of variability. This is a scatter plot of what our generated 
dataset looks like: 


Step 2: Write and execute the neural network training script 


Write and execute the training script for the neural network code using the 
code in GitHub. You can see the Keras model is being used here. Additionally, 
some of the data is normalized within this code as well. As you learned in this 
chapter, normalization is a key step to ensure that each feature contributes 
proportionately to the model’s learning process. After executing the training 
script, we see the following: 


Os 12ms/step - accuracy: @.7323 - loss: 6.5313 - val_accuracy: 0.7500 - val_loss: 0.4898 


Os 4ms/step - accuracy: 0.8208 - loss: 0.4351 
Test loss: 0.4280831813812256 
Test accuracy: 0.824999988079071 


Figure 3.21: Network Training 


As you can see in this screenshot, the model completed 20 epochs, and the test 
loss was 0.4281 with an accuracy of 82.5%. This effectively accomplishes the 
goal of training a custom neural network from a dataset. We could take this 
further by predicting the injury risk from the provided data. 


Step 3: Write and execute prediction code 


Let’s re-load the neural network while also having it predict injury risk from a 
sedan with a 12mph delta-V (collision severity) and 55g peak acceleration. 
The following output is observed after the model is re-trained: 


1/1 Os 55ms/step 


Predicted Injury Risk for Sedan with delta-V of 12 mph, 55g acceleration: 0.64 


Figure 3.22: Prediction Output 
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Discussion 


Based on the trained data, the neural network predicts the injury risk from our 
sedan of 12mph dV and 55g acceleration to be 64%. This could be likened to 
the typical logistic regression type analysis typically conducted for these types 
of datasets. However, even in this simple feed-forward type of neural network 
we generated, it is clear how many variables could be included to train the 
network, including non-linear or variable data. Unlike logistic regression, 
neural networks can handle non-linear relationships and complex patterns in 
the data. There can be overfitting concerns and various other considerations 
already discussed in the chapter. This laboratory example shows a simplistic 
method for training a feedforward neural network for classification tasks. 


rt 
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Chapter 4 


Brainwaves Unleashed: EEG 
and MEG 


Introduction and Learning Objectives 


In this chapter, we will explore the critical role of Electroencephalography 
(EEG) and Magnetoencephalography (MEG) in neuroengineering. These non- 
invasive techniques are essential for recording and analyzing the brain’s 
electrical and magnetic activities. The chapter will provide a comprehensive 
understanding of the operational principles, hardware, and setup of EEG and 
MEG systems, as well as their applications and significance. By the end of this 
chapter, you will be able to 


Recognize the functional significance of EEG frequency bands and 
their association with cognitive states. 

Understand the setup and configuration procedures for accurate 
data acquisition. 

Describe the components and technical specifications of EEG and 


MEG systems. 

Explain the operational principles of MEG and its components. 
Identify noise sources in EEG and MEG recordings and strategies 
for minimizing them. 

Explain the software tools used for EEG and MEG studies’ signal 
processing and data analysis. 


Understanding the historical development of EEG and MEG can provide 
valuable context. EEG, first developed by Hans Berger in the 1920s, 
revolutionized brain research by allowing scientists to record electrical activity 
from the scalp. MEG, however, was introduced in the late 1960s and 1970s 
thanks to advancements in superconducting technology. Both technologies 
have evolved significantly, with EEG widely used in clinical and research 
settings and MEG providing unparalleled precision in mapping brain activity. 
This historical perspective shows how technological advancements have 
continually improved our ability to study the brain. 
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Figure 4.1: EEG Frequency Band Classifications [104] 


Classification of EEG 


Electroencephalography (EEG) is a noninvasive method for recording 
electrical activity in the brain. This technique is crucial in neuroengineering as 
it helps us understand various brain functions and states. EEG signals are 
classified by the frequency bands of the brain waves they capture. These 
frequency bands, Delta, Theta, Alpha, Beta, and Gamma, represent different 
types of neural activity and cognitive states [103]. 

EEG Frequency Bands: Delta, Theta, Alpha, Beta, Gamma 

EEG frequency bands are categorized based on their oscillation rates, 
measured in Hertz (Hz). Table 4.1 shows the different band types and 
associated brain functions. Fig. 4.1 [104] illustrates the different EEG 
frequency bands, showcasing the distinct waveforms related to Delta, Theta, 
Alpha, Beta, and Gamma bands over a five-second interval [103]. 
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Table 4.1: Band Types 


F(Hz) 


5-4 
a 
-13 


13-30 


Gamma |] 30-100 


Frequency Description 


Delta waves are the slowest and 
are typically observed during 
deep sleep. They play a role in 
restorative processes within the 
body and brain. 


Theta waves are prevalent during 
light sleep, deep meditation, and 
relaxation. 


Alpha waves are dominant when 
awake but relaxed, often with 
closed eyes. 


Beta waves are faster and are seen 
during active thinking, problem- 
solving, and focused mental 
activity. 


Gamma waves are the fastest and 
are associated with high-level 
information processing, 
including perception, 
consciousness, and cognitive 
functioning. 


Function 


Essential for restorative sleep and 
healing processes, delta waves 
facilitate deep restorative sleep 
and are crucial for physical and 
mental recovery. 
Linked with memory and 
learning, theta waves emerge 
during light sleep and deep 
meditation, facilitating memory 
consolidation and creative 
thinking. 


Associated with relaxation and 
stress reduction, alpha waves 
dominate during restful, wakeful 
states and indicate a calm and 
meditative mind. 


Representing active concentration 
and problem-solving, beta waves 
are prominent during _ tasks 
requiring focused mental effort 
and active thinking. 


Associated with peak cognitive 
functioning and information 
processing, gamma waves are 
involved in tasks requiring high- 
level cognitive functions, such as 
problem-solving, perception, and 
consciousness. 


Analytical Techniques: Band Power, Connectivity Analysis 


To extract meaningful information from EEG data, several analytical 


techniques are employed: 


Band Power 
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Figure 4.2: Topographic Maps of Band Power Distributions for Delta, Theta, Alpha, Beta, and 
Gamma Waves [105] 


Left Right 


This technique measures the power within specific frequency bands to quantify 
the intensity of brain activity. By analyzing band power, researchers can assess 
the activity level in different cognitive states, such as relaxation or 
concentration. Fig. 4.2 [105] depicts the band power distributions for Delta, 
Theta, Alpha, Beta, and Gamma waves, visually representing their relative 
intensities [106]. 

Connectivity Analysis 

Connectivity analysis examines the interactions between different brain 
regions. Coherence and phase synchronization help determine how different 


brain areas communicate and coordinate during various cognitive tasks. This 
analysis is crucial for understanding functional brain networks and their role 
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Figure 4.3: Connectivity and Dysconnectivity Patterns Across Different EEG Frequency 
Bands [95] 


in cognitive processes. Fig. 4.3 [95] shows the connectivity and 
dysconnectivity patterns in the brain for different EEG frequency bands, 
highlighting the communication pathways and their alterations in various 
states [108]. 

Classifying EEG into different frequency bands allows us to link specific brain 
wave patterns with various cognitive states and functions. Analytical 
techniques such as band power and connectivity analysis provide deeper 
insights into brain activity, enhancing our understanding of neural processes 
and advancing the field of neuroengineering. 

In addition to band power and connectivity analysis, time-frequency analysis 
(TFA) is a powerful technique used in EEG studies. TFA, including methods 
like wavelet transforms, allows researchers to see how the frequency content 
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Recording Computer 


Figure 4.4: Schematic Diagram of an EEG Machine [110] 


of a signal changes over time. This technique is especially useful for 
identifying transient events, such as short bursts of brain activity, which are 
crucial in understanding cognitive processes. TFA provides a detailed view of 
the brain's dynamic activity by decomposing EEG signals into constituent 
frequencies and observing their temporal evolution [109]. 


Components of EEG System 


An EEG system comprises several critical components that capture and 
analyze the brain's electrical activity. These components include electrodes, 
amplifiers, and data acquisition systems. Each element plays a unique role in 
ensuring accurate and reliable EEG recordings. The schematic diagram in Fig. 
4.4 [110] illustrates the overall configuration of an EEG machine used for 
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recording data from multiple participants, providing a visual overview of its 
main components and their interactions. 


Electrodes: Types and Materials 


Electrodes are the primary interface between the EEG system and the scalp, 
capturing the electrical signals generated by neuronal activity. There are 
various types of electrodes, each suited for specific applications: 

Surface Electrodes 

These are the most used in clinical and research settings. They are typically 
made from conductive materials such as silver/silver chloride (Ag/AgCl) or 
gold, ensuring efficient signal transmission. 

Needle Electrodes 

Used less frequently, these electrodes penetrate the skin to reach closv: to the 
source of electrical activity. They are made from stainless steel or platinum 
and are used in specific diagnostic procedures. 


Dry Electrodes 


These electrodes do not require conductive gel, making them suitable for quick 
setups and repeated use. They are often made from materials like titanium or 
conductive polymers. 

The electrode type and material choice depend on factors such as the required 
signal quality, the duration of recording, and patient comfort. 


Amplifiers: Function and Specifications 


Amplifiers are essential in EEG systems as they boost the weak electrical 
signals picked up by the electrodes, making them suitable for analysis. The 
basic functions and specifications of amplifiers include: 

Signal Amplification 

Amplifiers increase the amplitude of the electrical signals from the brain, 
which are typically in the microvolt range. 

Noise Reduction 


High-quality amplifiers are designed to minimize electrical noise and 
interference, ensuring a clear and accurate signal. 
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Frequency Response 

Amplifiers must have an appropriate frequency response to capture the wide 
range of brain wave frequencies, from delta to gamma waves. 

Channel Capacity 


The number of channels an amplifier supports is crucial, as it determines how 
many electrodes can be simultaneously recorded. 


Data Acquisition Systems 


Modern data acquisition systems (DAS) for EEG integrate seamlessly with 
software platforms that enhance user experience. User-friendly interfaces 
allow researchers to set up and monitor experiments easily. Advanced software 
tools offer real-time data visualization, automated artifact detection, and 
sophisticated analysis capabilities. This integration ensures that data collection 
is efficient and the resulting data can be quickly and accurately analyzed, 
facilitating faster research and clinical practice advancements. 

Critical features of DAS include: 


Analog-to-Digital Conversion (ADC) 

This process involves converting continuous electrical signals into discrete 
digital values, enabling computerized analysis. 

Sampling Rate 

The sampling rate must be sufficiently high to capture the nuances of brain 
activity. Typically, EEG systems use sampling rates ranging from 250 Hz to 
several kHz. 

Data Storage and Transfer 


Modern DAS can store large amounts of data and transfer it to computers or 
cloud-based systems for real-time analysis and long-term storage. 
Setup and Configuration 


Proper setup and configuration of the EEG system are vital for obtaining 
accurate and reliable recordings. This involves careful electrode placement and 
scalp preparation. 
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Figure 4.5: The 10-20 System for Electrode Placement in EEG [111] 


Electrode Placement 

Electrode placement is guided by standardized systems such as the 10- 20 
system, which ensures consistent and replicable results. 

Key points include 

Anatomical Landmarks 

Electrodes are placed on the scalp based on specific anatomical landmarks, 
ensuring standardized individual positioning. 

Number of Electrodes 

Depending on the study or clinical requirement, the number of electrodes can 
vary from a few (e.g., 8 or 16) to a full cap with 64 or more electrodes. 
Placement Techniques 


Techniques include using conductive gel or paste to enhance signal quality and 
secure the electrodes. 
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Accurate electrode placement is crucial for obtaining high-quality EEG data. 
Misplaced electrodes can lead to incorrect data interpretation and affect the 
reliability of the results. For example, improper placement may miss key brain 
regions associated with specific cognitive tasks, leading to incomplete or 
misleading conclusions. 


Scalp Preparation 


Scalp preparation is crucial for reducing impedance and improving signal 
quality. This process typically involves: 


Cleaning the Scalp 


The scalp is cleaned with an abrasive gel or solution to remove oils and dead 
skin cells, reducing the impedance between the scalp and electrodes. 


Applying Conductive Gel 


Conductive gel ensures a good connection between the electrodes and the 
scalp, facilitating efficient signal transmission. 


The 10-20 System 


The 10-20 system is a standardized method for placing electrodes on the scalp 
in EEG. This system ensures consistent and reproducible electrode placements 
across individuals and studies, facilitating reliable data collection and analysis 
[96]. The 10-20 system refers to electrode positioning based on the relative 
distances between specific anatomical landmarks on the scalp. The name "10- 
20" indicates that the distances between adjacent electrodes are 10% or 20% 
of the skull's total front-back or right-left distance. This method ensures that 
electrode placement is systematic and standardized. Electrodes are placed at 
intervals of 10% or 20% of the total length of the scalp, measured from the 
nasion (the bridge of the nose) to the inion (the bony bump at the back of the 
head) and from the left to the right preauricular points (just above the ears). 
Key positions include: 

e Fpl, Fp2: Frontopolar positions 

e Fz, Cz, Pz, Oz: Midline positions from front to back 

e F3, F4, C3, C4, P3, P4, Ol, O2: Lateral positions over the frontal, 

central, parietal, and occipital lobes 
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This systematic approach ensures that EEG recordings are consistent and 
comparable across sessions and subjects. The diagram in Fig. 4.5 [111] 
illustrates the 10-20 system, highlighting the placement of electrodes based on 
specific percentages of the head's total circumference and dimensions. 


Signal Resolution and Sampling Rates 


Signal resolution in EEG refers to the system's ability to detect and distinguish 
small changes in electrical activity on the scalp, which is crucial for identifying 
subtle brain processes. The higher signal resolution allows for a more precise 
and detailed representation of neural activity, which is essential for accurate 
analysis. Higher-resolution EEG recordings show more defined and distinct 
waveforms, capturing finer details of neural events. Conversely, lower- 
resolution recordings appear more blurred and less precise, potentially missing 
critical neural signals. This distinction underscores the importance of high- 
quality electrodes, amplifiers, and signal processing in achieving optimal 
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Figure 4.6: MATLAB plot of EEG sampling rates. Red circles represent the sampled data 
points, while the black line represents the actual signal 
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signal resolution in EEG systems. We will discuss this more later in this 
chapter when we talk about noise in signals. 

Fig. 4.6 illustrates the concept of sampling rates in EEG. The sampling rate 
refers to the frequency at which the EEG system records data points per 
second. In the image, two different sampling rates are depicted: 

1. Panel (A): This panel shows EEG data sampled at a lower rate. The red 
circles represent the recorded data points. Due to the lower sampling 
rate, the recorded data points are sparse and miss some details of the 
actual waveform (black line). This can result in aliasing, where the 
actual signal is not accurately represented. 

2. Panel (B): This panel shows that the EEG data was sampled at a higher 
rate. The red circles, representing the recorded data points, are much 
closer together, capturing the waveform's finer details. This higher 
sampling rate provides a more accurate and detailed representation of 
the neural activity. 

Higher sampling rates allow for a more detailed and accurate representation of 
neural activity, capturing rapid changes and transient events. Conversely, 
lower sampling rates may miss these details, leading to a less precise signal 
representation. This distinction is crucial in EEG to ensure the accurate 
detection and analysis of brain activity, especially for studies involving fast 
neural dynamics. 


MEG Operational Principles 


Magnetoencephalography (MEG) is a non-invasive neuroimaging technique 
used to measure the magnetic fields produced by neuronal activity in the brain. 
Unlike electroencephalography (EEG), which detects electrical activity, MEG 
captures the magnetic signals, providing high temporal and spatial resolution. 
This makes MEG an invaluable tool for studying brain function and 
diagnosing neurological disorders. 


Biomagnetic Signal Detection 

Biomagnetic signal detection in MEG involves capturing the minute magnetic 
fields generated by the electrical currents of active neurons. These magnetic 
fields are fragile, often in the femtocell range, and require susceptible detectors 
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known as Superconducting Quantum Interference Devices (SQUIDs) to be 
accurately measured. 


Setup Considerations 

Setting up an MEG system involves several critical considerations to ensure 
optimal performance and accurate data acquisition. These include the room 
and environmental requirements, system calibration, and ensuring all 
components are correctly installed and maintained. 

Components of the MEG System 

Fig. 4.7 [112] illustrates the key components of an MEG system, specifically 
focusing on the SQUID (Superconducting Quantum Interference Device) and 
its associated circuitry. Each element is critical in accurately detecting and 
measuring the brain's magnetic fields [113]. 

Pickup Coil 

The pickup coil is the initial component that detects the magnetic fields 
generated by neuronal activity. These coils are sensitive to the tiny magnetic 
signals produced by the brain. 

Input Coil 

The input coil is connected to the pickup coil and transfers the detected 
magnetic signal to the SQUID. 

SQUID 

The SQUID is at the core of the MEG system. It is susceptible and can detect 
minimal changes in magnetic fields. The SQUID operates at cryogenic 
temperatures, maintained by liquid helium (LHe), to ensure superconducting 
properties. 

Feedback Coil 

The feedback coil works with the SQUID to maintain a stable measurement 


environment by applying feedback current to cancel out the detected magnetic 
fields. This helps keep the SQUID's linearity and sensitivity. 
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Heater 

The heater occasionally warms the SQUID to reset its superconducting state if 
it becomes saturated. 

Pre-amplifier 

The pre-amplifier boosts the signal from the SQUID before further processing. 
This is crucial as the signals detected are feeble and need amplification for 
accurate measurement. 

Integrator 

The integrator processes the amplified signal, smoothing out the fluctuations 
and providing a continuous output signal. 


Voltage Offset 


This component adjusts the signal's baseline voltage to ensure accurate read- 
ings and compensation for any drifts in the signal. 
Feedback Resistor 


The feedback resistor helps control the feedback current applied to the 
feedback coil, which is essential for the measurement's stability and accuracy. 
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Figure 4.7: Block Diagram of Key Components in a MEG System a) A single channel axial 
gradiometer with SWID inside a dewar filled with liquid helium b) Flux Transformer and 
SQUID c) Axial and planar gradiometers [112] 
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Bias Flux and Bias Current 


These provide the necessary operating conditions for the SQUID, ensuring it 
functions within its optimal parameters. 


FLL (Flux Locked Loop) Sw itch 
This switch helps maintain the SQUID in a flux-locked state, ensuring linearity 
and sensitivity in the measurements. 


Reset Switch 


The reset switch resets the SQUID if it exceeds its operational range, ensuring 
continuous and accurate measurements. 


Room and Environmental Requirements 

The MEG system must be in a magnetically shielded room to prevent external 
magnetic interference. These rooms typically use multiple layers of metal and 
aluminum to protect against environmental noise. Additionally, the room 
should be acoustically insulated to minimize noise affecting sensitive 
measurements. Environmental controls, such as temperature and humidity 
regulation, are essential for optimal operating conditions. 


Event-Related Potentials (ERPs) 


Event-related potentials (ERPs) are the brain's electrical responses directly 
relating to specific sensory, cognitive, or motor events. These responses are 
time-locked, meaning they are consistently triggered at specific times after a 
particular event. ERPs are typically measured using electroencephalography 
(EEG), which records electrical activity from the scalp. 

Two main features characterize ERPs: 

- Latency: The timing of the ERP response, measured in 
milliseconds, indicates how quickly the brain responds to the 
event. 

- Amplitude: The strength or size of the electrical response, 
which reflects the intensity of the brain's activity related to 
processing the event. 

Fig. 4.8 [114] illustrates the various components of ERPs, highlighting the 
different waves and their associated cognitive functions: 
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e N1 Component: An early negative wave associated with initial sensory 
processing. 

e P2 Component: A positive wave following N1, related to early 
perceptual processing. 

e N2 Component: A negative wave involved in cognitive control and 
conflict monitoring. 

e P3 Component: A positive wave linked to attention and decision- 
making processes. 

e N400 Component: A negative wave associated with language and 
semantic processing. 

e Late Positive Component (LPC): A late positive wave involved in 
memory and higher-order cognitive processes. 
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Figure 4.8: Components of ERP Sourced [114] 
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Types of ERPs 
P300 
Associated with attention and decision-making, typically elicited in oddball 


paradigms where subjects detect infrequent targets among frequent non- 
targets. 


N400 


Linked to language processing and semantic incongruities, observed when a 
word or sentence is semantically unexpected. 

ERN (Error-Related Negativity) is related to error detection and response 
monitoring and occurs when a person makes a mistake. 


Applications in Research and Clinical Practice 

ERPs and MEG have significant applications in both research and clinical 
practice. The image illustrates the combined use of MEG and EEG in clinical 
settings, highlighting their complementary strengths in diagnosing and 
monitoring neurological conditions. 
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Figure 4.9 Combined EEG and MEG in Clinical Practice [115] 
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Cognitive Function Analysis 

ERPs are extensively used in cognitive neuroscience to study brain processes 
such as perception, attention, memory, and language. Researchers can infer the 
neural mechanisms underlying various cognitive functions by analyzing ERP 
components’ timing (latency) and amplitude. For example, the P300 
component is used to study attention and working memory, while the N400 
component is used to investigate language comprehension and semantic 
processing. With its high spatial resolution, MEG complements ERP studies 
by providing precise localization of neural activity, offering a comprehensive 
understanding of brain function. Fig. 4.9 [115] demonstrates the application of 
MEG and EEG in clinical practice, showing their utility in diagnosing and 
monitoring neurological conditions [116]. Each part of the image is explained 
below: 

1. MEG Recordings (Top Left): This section shows the MEG signals 
recorded from various sensors around the head. These signals capture 
the magnetic fields generated by neuronal activity. The displayed 
waveforms represent brain activity over time, indicating areas of 
interest where abnormal activity may be present. 

2. EEG Recordings (Bottom Left): This section displays EEG signals 
recorded from electrodes on the scalp. EEG captures the electrical 
activity of the brain, providing high temporal resolution. The 
waveforms here show brain activity and are used to identify abnormal 
patterns associated with neurological conditions. 

3. MEG Activity in Selected Channel (Top Center): This part of the 
image zooms in on a specific channel's MEG activity, providing a 
detailed view of the magnetic field changes over time. This focused 
view helps analyze the characteristics of neural responses in a specific 
brain region. 

4. Top View of MEG Sensors (Top Right): This diagram shows the 
layout of MEG sensors positioned around the head. Each sensor's 
location is mapped to ensure comprehensive brain coverage, allowing 
for precise localization of magnetic signals. 
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Region of Interest (Top Right Box): Highlighted within the top view 
of MEG sensors, this box indicates the specific area of the brain being 
closely examined. It shows the distribution of 

MEG signals within this region help to pinpoint areas of abnormal 
activity. 

MEG Field Distribution (Center Right): This section visualizes the 
magnetic field distribution detected by MEG sensors. The field lines 
and arrows represent the direction and intensity of the magnetic fields, 
providing insights into the underlying neural activity. 

Co-registration of MEG Dipole with Patient's MRI Brain (Bottom 
Right): This set of images shows the co-registration of MEG data with 
the patient's MRI scans. Combining MEG's functional data with MRI's 
structural information allows for precise localization of neural sources 
within the anatomical context. This is particularly useful in pre- 
surgical planning for epilepsy patients, where accurate localization of 
epileptic foci is critical. 
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Figure 4.10: Illustration showing environmental interference affecting EEG and MEG record- 
ings. Sources like power lines, electronic devices, and the Earth's magnetic field are depicted 
as waves or signals interfering with EEG and MEG data. Arrows indicate the interference 
direction towards the EEG and MEG devices, highlighting the impact of these external 
sources on the recordings. Image generated with AI. 
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By examining each part of the image, we can see how MEG and EEG 
recordings, along with their integration with MRI data, provide comprehensive 
insights into brain function and abnormalities. This multi-modal approach 
enhances the diagnosis and monitoring of neurological conditions, facilitating 
better clinical outcomes. Fig. 4.10 shows common environmental interferences 
affecting EEG and MEG recordings. 


Noise Considerations 
Sources of Noise in EEG and MEG 


Environmental Interference: EEG and MEG systems are susceptible to 
environmental interference from external sources such as power lines, 
electronic devices, and the Earth's magnetic field. This noise can significantly 
distort the recorded signals, making it difficult to interpret the underlying 
neural activity accurately. 

Biological Artifacts: Biological artifacts are significant noise sources in EEG 
and MEG recordings, originating from physiological activities such as eye 
movements, blinks, muscle activity, and cardiac signals. Fig. 4.11 [106] 
illustrates how these artifacts combine with the accurate EEG signal to form 
the observed EEG data. Eye movements and blinks produce low-frequency 
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Figure 4.11: Illustration of Biological Artifacts Affecting EEG Signal - The image shows how 


various sources of biological artifacts, including eye movements, eye blinks, muscle activity, 
and cardiac signals, combine with the true EEG signal to form the observed EEG data. [106] 
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drifts and rapid spikes, respectively, while muscle activity generates high- 
frequency noise, and cardiac signals create rhythmic pulses. These artifacts 
can significantly distort the EEG signal, complicating the interpretation of 
neural activity. Understanding these noise sources is crucial for applying 
artifact rejection and filtering techniques to enhance the accuracy of EEG and 
MEG recordings. 

Strategies for Minimizing Noise 

Shielding Techniques: To combat environmental interference, MEG systems 
are typically housed in magnetically shielded rooms. These rooms are 
constructed with mu-metal and aluminum to block external magnetic fields. 
For EEG, proper grounding and shielded cables can help reduce electrical 
interference. 

Signal Processing Approaches: Advanced signal processing techniques are 
employed to filter out noise and enhance the quality of the recorded data. This 
includes using digital filters to remove unwanted frequencies and algorithms 
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Figure 4.12: Comparison of Raw and Notch Filtered EEG Signals [118] 
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to identify and reject artifacts. Techniques such as Independent Component 
Analysis (ICA) can be particularly effective in separating neural signals from 
noise. 


MEG/EEG Software 


Signal Processing Tools 

Filtering 

Filtering is a fundamental signal processing tool that removes noise from EEG 
and MEG data. High-pass filters can eliminate low-frequency noise, while 
low-pass filters remove high-frequency noise [117]. Band-pass filters are used 
to isolate specific frequency bands of interest. 

Artifact Rejection 

Artifact rejection algorithms identify and remove non-neural signals from the 
data. Techniques such as Independent Component Analysis (ICA) and 
Principal Component Analysis (PCA) can separate artifacts from neural 
signals, improving the clarity and accuracy of the recordings. 

Fig. 4.12 [118] shows EEG recordings from the left and right frontal areas, 
comparing raw data (left panels) with notch-filtered data (right panels). Notch 
filters remove specific frequencies, such as power line noise (e.g., 50/60 Hz), 
which can significantly distort the EEG signal. The raw data panels display 
high noise levels, making it difficult to discern the underlying neural activity. 
In contrast, the notch-filtered panels show a cleaner signal with reduced noise, 
allowing for a more straightforward interpretation of neural activity. This 
demonstrates the effectiveness of filtering in enhancing the quality of EEG 
recordings by eliminating unwanted noise [118]. 


Analysis Software 

Time-Frequency Analysis 

Time-frequency analysis methods, such as wavelet transforms and short-time 
Fourier transforms, allow researchers to examine how the frequency content 
of neural signals changes over time. This is crucial for studying dynamic neural 
processes and understanding how different brain rhythms are involved in 
various cognitive functions. 

Source Localization Techniques 
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Figure 4.13: Brain Mapping for Real EEG Signals [119] 


Source localization techniques aim to pinpoint the origins of neural activity 
within the brain. Methods like beamforming and equivalent current dipole 
modeling use the recorded data to estimate the spatial location of neural 
sources, enhancing the spatial resolution of EEG and MEG. 


Visualization Tools 

Brain Mapping 

Brain mapping tools visually represent neural activity across different brain 
regions. These maps help researchers and clinicians understand the spatial 
distribution of brain functions and identify areas affected by neurological 
conditions. The image shows brain mapping for accurate EEG signals, 
highlighting areas of varying neural activity. The color scale indicates the 
intensity of the signals, with different colors representing different activity 
levels. This visual representation allows for easy identification of active brain 
regions and can be crucial for diagnosing and monitoring neurological 
disorders. Fig. 4.13 [119] illustrates the spatial distribution of neural activity 
across the brain, as captured by EEG recordings. The color scale, ranging from 
blue (low activity) to red (high activity), clearly visualizes which brain areas 
are most active. Brain mapping tools like this are essential for understanding 
different brain regions' complex functions and identifying abnormalities in 
clinical settings [120]. 

Real-Time Data Display 

Real-time data display tools provide immediate feedback on ongoing brain 
activity, allowing researchers to monitor experiments and adjust on the fly. 
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This capability is handy in clinical settings for real-time diagnostics and 
monitoring, enabling quick responses to changes in a patient's neural activity 
and facilitating immediate intervention when necessary. 


Advantages and Disadvantages 


EEG 


Pros: EEG is noninvasive, cost-effective, and offers high temporal resolution, 
making it suitable for studying rapid neural processes. Due to its accessibility 
and ease of use, it is widely used in research and clinical settings. 

Cons: EEG has lower spatial resolution than MEG, as the skull and scalp 
distort electrical signals. It is also susceptible to various artifacts from muscle 
activity and environmental noise, which can complicate data interpretation 
[121]. 


MEG 


Pros: MEG provides high spatial and temporal resolution, allowing for precise 
localization of neural activity. It is non-invasive and can detect minute 
magnetic fields generated by neuronal currents without affecting the skull or 
scalp. 

Cons: MEG systems are expensive and require specialized facilities, including 
magnetically shielded rooms and cryogenic cooling systems. The operational 
and infrastructure requirements can limit their accessibility compared to EEG 
[122]. 


Chapter 4: Summary 


In this chapter, we have delved into the fundamental roles of 
Electroencephalography (EEG) and Magnetoencephalography (MEG) in 
neuroengineering. These noninvasive techniques are crucial for recording and 
analyzing the brain's electrical and magnetic activities. They provide critical 
brain function insights and advance research and clinical practice. 

We started by exploring the importance of different EEG frequency bands and 
their associations with various cognitive states. Understanding the setup and 
configuration of EEG systems, including electrode placement and scalp 
preparation, is essential for acquiring high-quality data. We also examined 
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EEG systems' components and technical specifications, such as electrodes, 
amplifiers, and data acquisition systems, emphasizing their roles in ensuring 
accurate neural recordings. In detail, we discussed the operational principles 
of MEG, including biomagnetic signal detection and system setup 
considerations. We highlighted the essential components of MEG systems, 
such as SQUID sensors and cooling systems, and the room and environmental 
requirements necessary for optimal performance. Event-related potentials 
(ERPs) were explained, focusing on their definitions, components, and 
applications in cognitive function analysis and clinical diagnosis. Integrating 
MEG and EEG in clinical practice enhances the diagnosis and monitoring of 
neurological conditions, providing comprehensive insights into brain function 
and abnormalities. We also covered noise considerations, including 
environmental interference and biological artifacts, along with strategies for 
minimizing noise through shielding techniques and advanced signal 
processing approaches. The significance of MEG/EEG software tools, 
including filtering, artifact rejection, time-frequency analysis, and source 
localization techniques, was emphasized in improving the clarity and accuracy 
of neural recordings. Visualization tools like brain mapping and real-time data 
display further aid in understanding neural activity and facilitating clinical 
diagnostics. Finally, we evaluated the advantages and disadvantages of EEG 
and MEG. EEG is non-invasive, cost-effective, and offers high temporal 
resolution but is limited by low spatial resolution and susceptibility to artifacts. 
MEG provides high spatial and temporal resolution but requires expensive and 
specialized facilities. 

As you conclude this chapter, I encourage you, as a neuroengineering 
professional, to seize the opportunity to impact EEG and MEG significantly. 
Your innovative research, technological advancements, and_ clinical 
applications can drive forward our understanding of brain function, improve 
diagnostic and therapeutic outcomes, and contribute to the development of 
new tools and methodologies. Embrace the challenges and possibilities within 
this field and strive to push the boundaries of what we know about the brain, 
ultimately making a profound difference in science and healthcare. The future 
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of neuroengineering is bright, and your contributions can shape its course and 
legacy. 
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Chapter 4: Learning activities 


Learning Activity 4.1 


A concept map is a visual representation tool to organize 
and structure knowledge. It displays relationships between 
concepts in a graphical format, typically through nodes and 
connecting lines. Each node represents a concept, and lines 


(often labeled) illustrate the connections or relationships between these 


concepts. 


Key Features of a Concept Map: 


1. 
2. 


Nodes are usually circles or boxes containing key concepts or ideas. 
Links: Lines or arrows that connect the nodes, showing how the con- 
cepts are related. 

Labels: Words or phrases on the links describing the relationship be- 
tween the connected concepts. 

Hierarchy: Concept maps often have a hierarchical structure, with 
more general or broader concepts at the top and more specific or de- 
tailed concepts branching out below. 

Cross-Links: These show relationships between concepts in different 
parts of the map, illustrating the complexity and interconnectivity of 
the ideas. 


Benefits of Concept Maps: 


1. 


Visual Learning: They help visualize the relationships between con- 
cepts, making complex information easier to understand. 
Organization: They organize information in a clear, structured way, 
aiding memory and recall. 

Critical Thinking: Creating concept maps requires critical thinking to 
determine the relationships and hierarchy among concepts. 
Collaboration: They can be used collaboratively, with multiple people 
contributing to and discussing the map. 

Problem-Solving: They help identify gaps in knowledge and can be 
used to plan and solve problems systematically. 
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Activity 
Individual activity: draw a concept map connecting the different 'brain' nonin- 
vasive continuous signals. 


Learning Activity 4.2 


Project-Based Learning (PBL) is an educational approach 
that emphasizes hands-on, student-centered learning 
through extended projects that tackle real-world problems 


and challenges. This method engages students in active 
exploration and inquiry, allowing them to apply knowledge and skills 
practically and meaningfully. Projects are designed to be relevant to students' 
lives and interests, promoting deeper understanding and _ sustained 
engagement. Students often work in teams, fostering collaboration and 
communication skills, and they can take ownership of their learning by making 
choices about their project's direction and outcome. PBL is typically 
multidisciplinary, integrating various subjects and helping students see the 
connections between different areas of knowledge. The learning process 
culminates in a public product or presentation, allowing students to share their 
work with a broader audience. Reflective practice is also a key component, 
with students and teachers regularly assessing the learning outcomes and the 
learning process. PBL enhances critical thinking, problem-solving, and 
teamwork, making learning engaging and impactful. 

Activity: propose the following: as a student engineer, you need to design a 
machine capable of generating EEG data; how would you do it? 

Step 1: form teams of 3 students 

Step 2: discuss with the rest of the class how many different approaches were 
found. 
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Learning Activity 4.3 


Similar to the previous activity: "PBL: a pilot is getting 
drowsy, you are asked as an engineer to address the issue; 
how can you use what you have learned so far to create a 


device that solves the problem? Show an FBD" 
Step 1: form teams of 3 students 

Step 2: discuss with the rest of the class how many different approaches were 
found. 


@ 


® 
PAs 
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Chapter 4: Lab introduction 


In this series of lab exercises, you will gain hands-on experience with 
EEG signal processing and analysis using MATLAB. These labs are 
designed to apply your understanding of electroencephalography 
(EEG) data processing through practical applications. 

In the first lab, you will follow a step-by-step laboratory exercise 
using MATLAB to perform EEG signal processing. Utilizing the 
Signal Processing Toolbox, you will generate and analyze EEG 
signals, simulate different brain states, add noise, amplify signals, 
apply filters, and visualize the results. By the end of this lab, you will 
be proficient in handling EEG data and using various signal- 
processing techniques to interpret brain activity. 

The second lab introduces you to EEGLAB, a MATLAB toolbox 


specifically designed for processing EEG and MEG. Distributed 


under the free BSD license, EEGLAB provides a comprehensive 
suite of tools for displaying and analyzing EEG data. You will 
become familiar with the toolbox’s capabilities, learning to process 
and visualize complex electrophysiological data effectively. This lab 
will equip you with essential advanced EEG data analysis skills, 
preparing you for further research and applications in neuroscience 
and biomedical engineering. 
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Chapter 4: Lab Example 1 


Following this step-by-step laboratory exercise, MATLAB 
will be used to perform the EEG signal processing, as 
illustrated in your example. You will need the Signal 


Processing Toolbox to complete this lab. By the end of the 
exercise, you should be able to generate and analyze EEG signals, simulate 
different brain states, add noise, amplify signals, apply filters, and visualize 
the results. 


EEG Signal Processing in MATLAB 

Objective 

This exercise aims to understand and analyze EEG signals by generating 
synthetic EEG signals, simulating different brain states, adding noise, 
amplifying signals, filtering, and visualizing the results. 

Materials Needed 


- MATLAB software 
- Signal Processing Toolbox 
- Computer with sufficient computational capabilities 


Exercise Instructions 


Step 1: Set Up the MATLAB Environment 


1. Open MATLAB. 
2. Create a new script and name it ‘EEG Signal Processing.m’. 


Step 2: Define the Parameters 


Fs = 1000; % Sampling frequency in Hz 
T= 10; % Duration of the signal in seconds 
t = 0:1/Fs:T-1/Fs; % Time vector 


Step 3: Generate Awake and Drowsy EEG Signals 


io} 


% Generate awake EEG signal with alpha and beta 
waves 
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awake alpha wave = 0.7 * sin(2*pi¥*10*t); 
awake beta_wave = 0.5 * sin(2*pi*20*t) ; 
awake eeg signal = awake alpha wave + 


awake beta wave; 

% Generate normal drowsy EEG signal with lower 
amplitude alpha and beta waves 

drowsy alpha wave = 0.3 * sin(2*pi*10*t) ; 
cheowsyy lose! wens S 1). 2 3 palin ((2tjoul/7(0i7e)) 6 
drowsy eeg signal = drowsy alpha _ wave + 
drowsy beta wave; 


Step 4: Simulate Epileptiform Activity in Drowsy EEG 


epileptiform start = 3; % Time point (in seconds) 
to start epileptiform activity 
epileptiform duration = 1; % Duration of 


epileptiform activity 
epileptiform amplitude = 1.5; % Amplitude of 
epileptiform spikes 


epileptiform signal = zeros(size(t)); 

epileptiform indices = find(t >= epileptiform start 
(3 te <S (joa sjoieaicoram Siecle 
epileptiform duration) ); 
epileptiform signal (epileptiform indices) = 
epileptiform amplitude * 
SCMALS ((AVjoalte st (ic (Sjoal lleyoielicoiim sinless) — 
epileptiform start)); 


drowsy eeg with epileptiform = drowsy eeg signal + 
epileptiform signal; 


Step 5: Add Noise to Simulate Real-World EEG Signals 


noise level = 0.1; 
noisy awake alpha = awake alpha wave + noise level 
* randn(size(t)); 
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noisy awake beta = awake beta wave + noise level * 
randn(size(t)); 

noisy drowsy alpha = drowsy alpha wave + 

noise level * randn(size(t)); 
noisy drowsy beta = drowsy beta wave + noise level 
* randn(size(t)); 
noisy drowsy with epileptiform = 

drowsy eeg with epileptiform + noise level * 
randn(size(t)); 


ie} 


% Combine noisy signals 

awakeEEGSignal = noisy awake alpha + 
noisy awake beta; 

drowsyEEGSignal = noisy drowsy alpha + 
noisy drowsy beta; 
drowsyEEGWithEpileptiformSignal = 
noisy drowsy with epileptiform; 


Step 6: Amplify the Signals 


amplificationFactor = 2; 
amplifiedAwakeSignal = awakeEEGSignal * 
amplificationFactor; 
amplifiedDrowsySignal = drowsyEEGSignal * 
amplificationFactor; 
amplifiedDrowsyWithEpileptiformSignal = 
drowsyEEGWithEpileptiformSignal * 
amplificationFactor; 


Step 7: Design and Apply a Low-Pass FIR Filter 


Fe = 30; % Cutoff frequency 
order = 100; % Filter order 


% Design a low-pass FIR filter 
d = designfilt('lowpassfir', 'FilterOrder', order, 
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"CutoffFrequency', Fc, 'SampleRate', 
Fs, 

"DesignMethod', 'window', 'Window', 
"hamming'); 


6 Filter the amplified signals 
filteredAmplifiedAwakeEEG = filter(d, amplifiedA- 
wakeSignal) ; 

filteredAmplifiedDrowsyE! 
DrowsySignal); 
filteredAmplifiedDrowsyWithEpileptiformEEG = fil- 
ter(d, amplifiedDrowsyWithEpileptiformSignal) ; 


GI 


G = filter(d, amplified- 


Step 8: Apply Zero-Phase Filtering 


filteredAmp| ifiedAwakeEEG zeroPhase = filtfilt(d, 
amplifiedAwakeSignal) ; 
filteredAmp| ifiedDrowsyEEG zeroPhase = filtfilt(d, 
amplifiedDrowsySignal) ; 
filteredAmplifiedDrowsyWithEpileptiformEEG zero- 
Phase = filtfilt(d, amplifiedDrowsyWithEpilepti- 
ig@uemMsysLepaveltl)) 5 


Step 9: Plotting the Results 


Fo = 30; % Cutoff frequency 
order = 100; 6 Filter order 


% Design a low-pass FIR filter 
designfilt('lowpassfir', 'FilterOrder', order, 


Q 
I 


"CutoffFregquency', Fc, 'SampleRate', 
Fs, 

"DesignMethod', '‘window', 'Window', 
"hamming'); 


fe} 


% Filter the amplified signals 
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aa 
amp 
fil 
amp 
ell 
fil 


Step 10: Apply Zero-Phase Filtering 


amp 


teredAmplifiedAwakeEEG = filter(d, 
lifiedAwakeSignal) ; 

teredAmplifiedDrowsyEEG = filter(d, 
lifiedDrowsySignal); 
teredAmplifiedDrowsyWithEpileptiformEEG 

ter(d, amplifiedDrowsyWithEpileptiformSignal) ; 


Step 11: Plotting the Results 


(ey 


yAx 


adjustment based on actual signal ampli 


IESLGj 


% Awake EEG Signal 
sinleya Lote (4, 81, iL) & 


plo 
AL 
xla 


isRange 


ure; 


t(t, awake 
le('Awake 
bel ('Time 


yla 


bel ('Ampli 


y 


seep Sig oi) 


iS) 


EEGSignal); 


(a) )¢ 


teudie! )r- 
lim(yAxisRange) ; 


subplot (4,3,4); 


5 Define a common y-axis range for all 


a ° 
1 se) 


EEG Signal'); 


This range 


plot(t, amplifiedAwakeSignal) ; 
title('Amplified Awake Signal'); 


xla 


bel ('Time 


(Ss) “) sé 
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lteredAmplifiedAwakeEEG zeroPhase = filtfilt(d, 
lifiedAwakeSignal) ; 
lteredAmplifiedDrowsyEEG zeroPhase = filtfilt(d, 
lifiedDrowsySignal) ; 
teredAmplifiedDrowsyWithEpileptiformEEG zeroPhas 
ial Ibe ical dle (el, 
ifiedDrowsyWithEpileptiformSignal) ; 


plots 
might need 
tudes 


ylabel ('Amplitude'); 
ylim(yAxisRange) ; 


Sulplot (4a i); 

plot(t, filteredAmplifiedAwakeEEG) ; 
title('Filtered Amplified Awake Signal"); 
s<ilaloSil (7 waime (s)) /)) ¢ 

ylabel ('Amplitude'); 

ylim(yAxisRange) ; 


siuloyodhort (4 psp IU) § 

plot(t, filteredAmplifiedAwakeEEG zeroPhase) ; 
title('Zero-Phase Filtered Awake Signal"); 
silaloSil (Cwsaime Cs) Vs 

ylabel ('Amplitude'); 

ylim(yAxisRange) ; 


fe} 


% Normal Drowsy EEG Signals 
siulojollor (4,5 ,2)) § 

plot(t, drowsyEEGSignal) ; 
title('Normal Drowsy EEG Signal'); 
xlabel ("Time (s)"); 
ylabel ('Amplitude'); 
ylim(yAxisRange) ; 


Sulojolhorr (4,3 ,5)) 5 

plot(t, amplifiedDrowsySignal) ; 
title('Amplified Drowsy Signal'); 
Xlabel('Time (s)'); 

ylabel ('Amplitude'); 
ylim(yAxisRange) ; 


SUloOLOIE (4 pS pi!) B 

plot(t, filteredAmplifiedDrowsyEEG) ; 

title('Filtered Amplified Drowsy Signal"); 
] Time (s)'); 

ylabel ('Amplitude'); 

ylim(yAxisRange) ; 


sulojolbort (4 5 3, 1.1L) 5 


187 


plot(t, filteredAmplifiedDrowsyEEG zeroPhase) ; 
title('Zero-Phase Filtered Drowsy Signal"); 
leis (7 Waime (5) )\¢ 

ylabel ('Amplitude'); 

ylim(yAxisRange) ; 


fo} 


% Drowsy EEG with Epileptiform Activity Signals 
siolloyollene (C4 5 3) 4.3!) 8 
plot(t, drowsyEEGWithEpileptiformSignal) ; 
title('Drowsy EEG with Epileptiform Activity'); 
MILEIOSIL (sims (ey) ")) g 

ylabel ('Amplitude'); 

ylim(yAxisRange) ; 


subplot (4,3,6); 

plot(t, amplifiedDrowsyWithEpileptiformSignal) ; 
title('Amplified Drowsy with Epileptiform Signal"); 
x<label(“time (Ss) )r 

ylabel ('Amplitude'); 

ylim(yAxisRange) ; 


SUubpilor (Ars, 2)r; 

jOL@Ne (ic, 
filteredAmplifiedDrowsyWithEpileptiformEEG) ; 
title('Filtered Amplified Drowsy with Epileptiform 
Salone “)) p 

xleloall (! ines (s)) ")\¢ 

ylabel ('Amplitude'); 

ylim(yAxisRange) ; 


siulloyolhoic (4). Sp 12) F 

jOLLGHE (ep 
filteredAmplifiedDrowsyWithEpileptiformEEG zeroPhas 
S) & 
title('Zero-Phase Filtered Drowsy with Epileptiform 
Ssepareul!)) ¢ 

NaLeloeil (wats (s)) ")) ¢ 

ylabel ('Amplitude'); 

ylim(yAxisRange) ; 
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sgtitle('EEG Signals Processing: Awake vs. Normal 
Drowsy vs. Drowsy with Epileptiform Activity'); 


EEG Signals Processing: Awake vs. Normal Drowsy vs. Drowsy with Epileptiform Activity 


Awake EEG Signal Normal Drowsy EEG Signal Drowsy EEG with Epileptiform Activity 
92 j @ 2} g2 
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Figure 4.14: EEG Signal Results 


Results 

Top Row: Original EEG Signals 

Awake EEG Signal: Shows the combination of alpha and beta waves. 

Normal Drowsy EEG Signal: Lower amplitude alpha and beta waves indicate 
a drowsy state. 

Drowsy EEG with Epileptiform Activity: A clear spike at the 3-second mark 
indicates epileptiform activity. 

Second Row: Amplified EEG Signals 

Amplified Awake Signal: The awake signal is amplified, making the wave 
patterns more distinct. 

Amplified Drowsy Signal: The drowsy signal is similarly amplified. 
Amplified Drowsy with Epileptiform Signal: Amplified version of the drowsy 
signal with more pronounced epileptiform spikes. 


189 


Third Row: Filtered Amplified EEG Signals 


Filtered Amplified Awake Signal: High-frequency noise is reduced, but the 
main characteristics of the awake signal are preserved. 

Filtered Amplified Drowsy Signal: Similar noise reduction is observed. 
Filtered Amplified Drowsy with Epileptiform Signal: Noise reduction while 
maintaining the epileptiform activity. 

Fourth Row: Zero-Phase Filtered Signals 

Zero-Phase Filtered Awake Signal: Provides a cleaner signal with preserved 
phase characteristics. 

Zero-Phase Filtered Drowsy Signal: Shows a clear, filtered version of the 
drowsy state. 

Zero-Phase Filtered Drowsy with Epileptiform Signal: Cleaned signal 
showing the epileptiform spikes clearly. 


Conclusion 


The provided MATLAB code and image demonstrate the steps in generating, 
amplifying, filtering, and visualizing EEG signals. Each stage highlights 
different aspects of signal processing, showing how noise can be mitigated and 
important signal characteristics preserved. This exercise is crucial for 
understanding how EEG data is processed and analyzed in practical 
applications. 


A 
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Chapter 4: Lab Example 2 


In this lab, we will familiarize ourselves with EEGLAB, a 
MATLAB toolbox designed to process 
electroencephalography (EEG), magnetoencephalography 
(MEG), and other electrophysiological data. Distributed 
under the free BSD license, EEGLAB offers a comprehensive suite of tools 
for displaying and analyzing EEG data. 


Installation and Main Features 


EEGLAB is hosted by the Swartz Center for Computational Neuroscience at 
UC San Diego. You can download the toolbox from this link. We extend our 
gratitude to UC San Diego for providing this valuable resource. 


Download and Installation 


The download and installation process is similar to that of other MATLAB 
toolboxes: 
1. Visit the EEGLAB download page. 
2. Follow the instructions provided to download the appropriate version 
for your system. 
3. Unzip the downloaded file to a convenient directory on your computer. 
4. Add the EEGLAB folder to your MATLAB path. 


Running EEGLAB 


After installation, you can launch EEGLAB by running the eeglab.m file. To 
do this: 
1. Open MATLAB. 
2. Navigate to the directory where you unzipped EEGLAB. 
3. Type eeglab in the MATLAB command window and press Enter. This 
action should open the EEGLAB graphical user interface (GUD), 
which will appear in Fig. 3.15. 
In the following sections of this lab, we will explore the main features of 
EEGLAB for displaying and processing EEG data. These features 
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0 Name Date modified 


7 test_data 7/24/2024 407 AM 
|) eeglab_dataset 7/24/2024 407 AM 
|) eeglab_data_epochs_ica.set 7/24/2024 4:07 AM 


Figure 3.15: EEGLAB GUI 


include data importing, preprocessing, visualization, and analysis tools 


essential for neuroengineering research. 


If you select File- load the existing dataset and browse to the sample data folder 
in the toolbox, you will see that EEGLAB is a powerful tool for processing 
and analyzing EEG data and associated events. While you can import your 
own EEG data, we will work with existing datasets included in the EEGLAB 


Type Size 


File folder 
SET File 345 KB 
SET File 225 KB 


package for this lab. To load a sample dataset, follow these steps: 


1. Go to the File menu. 
2. Select Load existing dataset. 


“A) EEGLAB v2024.0 


File Edit Tools Plot Study Datasets Help 


Figure 4.15: EEGLAB Data Set 
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3. Browse to the sample data folder within the EEGLAB toolbox 
directory. 

4. You will see several sample dataset options from which to choose. 
These sample datasets will help you familiarize yourself with the 
features and capabilities of EEGLAB (Fig. 4.15). 

In this lab, we will work with the eeglab_data.set sample dataset. 

As shown, the sample data contains over 30,000 frames per epoch, 32 channels 
per frame, and 154 events, and it is sampled at 128 Hz, meaning 128 samples 
per second (Fig. 4.16). 


Loading the EEG Data 


1. Select eeglab_data.set from the available options. 
2. This dataset contains EEG data with 32 channels, each labeled 
accordingly. ##Visualizing the Data Once the data is loaded, EEGLAB 


“A) EEGLAB v2024.0 - x 


File Edit Tools Plot Study Datasets Help » 


Figure 4.16: Sample Data for lab 
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a Scroll channel activities -- eegplotQ -- Continuous EEG Data = Oo 


Figure Display Settings Help 


< 


Chan. Time Value i 
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Figure 4.17: Channel Plots 


allows you to plot all the channels in the time domain to observe 
different events (Fig. 4.17). In this dataset: Square events are 
highlighted in green. RT events are highlighted in red. 


@ Figure 3: spectopo() = Oo x 
File Edit View Insert Tools Desktop Window Help » 
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Figure 4.18: PSD Plot 
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Power Spectral Density 


EEGLAB also provides tools to plot the power spectral density (PSD) of the 
EEG signals (Fig. 4.18). This feature is useful for identifying channels that 
may be faulty or need to be excluded from the analysis. 


Inspecting Specific Channels 


You can isolate a specific channel and visualize its properties using 
the pop_prop function. This feature enables detailed inspection of individual 
channels, facilitating quick and efficient EEG data analysis (Fig. 4.19). 


4 pop_prop() - Channel 1 properties = x 
File Edit View Insert Tools Desktop Window Help ™ 
O6Guas ao ka 
Channel 1 Continous data ae 
59 
2 
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| 
59 
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N Activity power spectrum 
ox 20 
= 
= 
= 105 
oD 
ne) 
* 
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— 
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3 -10E l L ! l 
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Figure 4.19: Isolating a channel 
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Figure 4.10: Advanced Features: Event Related Potentials 


Advanced Features 


EEGLAB is a robust tool with a variety of advanced features: 

e Filtering: EEGLAB allows you to apply frequency-delimited filters 
to the signals, which helps isolate specific frequency bands of interest. 

e Artifact Removal: You can use independent component analysis 
(ICA) to remove artifacts from the data. 

e Event-Related Potentials (ERP): EEGLAB enables the extraction 
and classification of event-related potentials (ERP), which are brain 
responses directly resulting from specific sensory, cognitive, or motor 
events (Fig. 4.20). By averaging the responses of all events, you can 
build a signature waveform that is distinctive of the response to a 
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ry Figure 2 = x 


File Edit View Insert Tools Desktop Window Help » 
OSGeas aoe re 
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Figure 4.21: Advance Features: Electrode Placement 


specific stimulus. These basic and advanced tools provided by 

EEGLAB allow for rapid and in-depth inspection and analysis of EEG 

data. 
EEGLAB provides advanced tools for cleaning EEG data, including the 
"Clean RawData" and "Artifact Subspace Reconstruction (ASR)" methods. 
These tools are essential for removing noise and artifacts from EEG 
recordings, which can interfere with data analysis and interpretation. By using 
these tools, you can significantly improve the quality and reliability of your 
EEG data. To use the "Clean RawData" tool in EEGLAB, start by ensuring 
EEGLAB is running and your dataset is loaded. Navigate to the Tools menu, 
then select Reject data using Clean RawData. This will open a dialog box 
where you can configure parameters for data cleaning. The default settings are 
usually sufficient, but you can adjust them based on your dataset's specific 
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needs. Click OK to run the tool to process the data and remove segments with 
high noise levels or artifacts. Go to the Tools menu again for the Artifact 
Subspace Reconstruction (ASR) tool and select Reject data using ASR. In the 
dialog box that appears, configure the key parameters, such as the cutoff 
threshold, which determines the level of artifact removal, and the window 
length, which defines the length of data segments to be processed. After setting 
these parameters, click OK to run ASR. This tool will identify and correct 
artifacts by reconstructing the affected data segments. 

Both tools are integral for cleaning EEG data in EEGLAB, effectively 
removing noise and artifacts to enhance data quality. As shown in Fig. 4.22, 
the signals in red have been identified as potential rejects from the dataset. 


ERPs 


An Event-Related Potential (ERP) is a measured brain response directly 
resulting from a specific sensory, cognitive, or motor event. ERPs are derived 
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Figure 4.22: Cleaning EEG data 
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Figure 4.23: Event-related potential in response to a visual stimulus as obtained by averaging 
the EEG signal of multiple stimulus presentations (S1, S2,... Sn). 


from electroencephalography (EEG) recordings and are used in cognitive 
neuroscience, psychology, and other fields to study brain function (Fig. 4.23 ). 


How ERPs are Obtained Based on EEG 


1. 


EEG Recording Electroencephalography (EEG) involves placing 
electrodes on the scalp to measure the electrical activity produced by 
neurons in the brain. The raw EEG signal is a complex mix of 
electrical activity from multiple sources, including both spontaneous 
brain activity and responses to specific events. 

Experimental Design To obtain ERPs, researchers design experiments 
where participants are exposed to specific stimuli (e.g., visual, 
auditory, or tactile). These stimuli are presented at known times and 
often repeated to obtain reliable measurements. 

Time-Locking to Events The EEG data is segmented into epochs, 
which are time-locked to the onset of the stimuli. Each epoch includes 
data from a short time window before and after the stimulus. 
Averaging Because the EEG signal contains a lot of noise from 
unrelated brain activity, muscle movements, and external sources, 
individual epochs are averaged across many trials. This averaging 
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process helps to isolate the consistent brain response to the stimulus, 
as random noise tends to cancel out while the signal related to the 
event remains. 

Baseline Correction Before averaging, a baseline correction is often 
performed. This involves subtracting the mean voltage of a pre- 
stimulus period (e.g., the time just before the stimulus is presented) 
from the entire epoch. This helps correct any slow drifts in the EEG 
signal. 

Resulting ERP Waveforms The result of averaging and baseline 
correction is a series of ERP waveforms reflecting the brain's electrical 
response to the event. These waveforms consist of positive and 
negative voltage deflections (peaks and troughs) occurring at different 
latencies (times) after the stimulus. 


Key Components of ERPs 


Latency: The time between the onset of the stimulus and the 
occurrence of a particular ERP component (peak or trough). This 
reflects the timing of neural processes. 

Amplitude: The magnitude of the voltage change associated with an 
ERP component can indicate the strength of the neural response. 
Polarity: Whether the ERP component is positive (upwards) or 
negative (downwards). 

Topography: The distribution of ERP amplitudes across the scalp can 
provide information about the source of the neural activity. 


Common ERP Components 


P300: A positive deflection around 300 milliseconds after stimulus 
onset, often associated with attention and decision-making processes. 
N400: A negative deflection peaking around 400 milliseconds, related 
to language processing and semantic incongruence. 

N170: A negative component around 170 milliseconds, associated 
with face perception. 
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Figure 4.24: EEG Data Epochs 


Example of ERP Analysis Process 


1. Record EEG data while a participant performs a task involving 
repeated stimuli. 

Segment the data into epochs around each stimulus presentation. 
Apply baseline correction to each epoch. 

Average the epochs to obtain the ERP waveform. 

Analyze the ERP components to interpret the neural processes 


we eo 


involved in the task. ERPs provide valuable insights into the timing 
and nature of cognitive processes by allowing researchers to examine 
the brain's electrical response to specific events in real-time. EEGLAB 
includes the extraction and examination of Event-Related Potentials 
(ERPs). To analyze ERPs using EEGLAB, researchers first import 
their raw EEG data into the software. They then preprocess the data, 
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typically involving filtering to remove noise, epoching the continuous 
EEG data into time segments around the events of interest, and 
performing baseline correction. Artifact rejection or correction 
methods are applied to remove noise from non-neural sources such as 
eye movements or muscle activity. Once clean epochs are obtained, 
they are averaged to isolate the ERP components related to specific 
stimuli or tasks. EEGLAB provides various tools for visualizing these 
ERP waveforms, including plotting the averaged ERP data and 
topographical maps to examine spatial distributions of the ERP 
components. Additionally, EEGLAB supports statistical analysis and 
comparison of ERP components across different experimental 
conditions. This comprehensive suite of tools within EEGLAB 
facilitates detailed ERP analysis, helping researchers interpret the 
neural mechanisms underlying cognitive processes. 


A 
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Chapter 5 


Hluminating the Brain: Adven- 
tures with fMRI and fNIRS 
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Introduction and Learning Objectives 


Leading up to this chapter, you will have learned about the anatomy and 
functions of the brain and its neurons. Neuroimaging techniques are critical 
tools that allow us to visualize these concepts in intriguing ways. Functional 
Magnetic Resonance Imaging (fMRI) and Near-Infrared Spectroscopy 
(fNIRS) are common methods. This chapter will explore these methods in 
detail and provide you with the information required to navigate and appreciate 
these tools within the field of engineering. By the end, you’ll be able to 
understand the following: 


Understand the principles of MRI, fMRI, NIRS, and fNIRS. 
Identify basic hardware components of the fMRI and fNIRS. 

. Understand software tools used to process and analyze fMRI and 
JSNIRS data 


Compare and contrast {MRI and {NIRS in terms of their applications 
in neuroengineering 

Apply knowledge of {MRI and {NIRS to practical neuroengineering 
laboratory exercises and research 


Functional Magnetic Resonance Imaging 
(fMRI) 


The Basics of MRI and {MRI 


At its core, MRI is a non-invasive neuroimaging technique that utilizes a large 
magnetic field along with radio waves to obtain a series of cross-sectional 
images of the brain (or any body part). Functional MRI builds on these 
principles by measuring changes in blood flow, specifically oxygenated blood, 
to visualize areas of increased brain activity. This approach, known as the 
Blood-Oxygen-Level Dependent (BOLD) signal, allows researchers to 
identify specific brain areas involved during various tasks [123]. 

For example, fMRI can show increased activity in the primary motor cortex 
when a subject performs a motor task, such as tapping their fingers. During a 
cognitive task, such as solving a complex math problem, fMRI could show 
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increased activity in the parietal lobe (and likely many other regions). Other 
interesting approaches to fMRI application involve the study of emotional 
responses, such as detecting heightened activity in the amygdala when viewing 
emotionally disturbing images. 

By capturing changes in specific brain regions, {MRI can provide invaluable 
insight into the brain's functional organization. This allows researchers to map 
out our brain's vast network of connections. Projects such as the Human 
Connectome Project are working toward this initiative because researchers are 
motivated to improve our understanding of the brain. Now, we will discuss the 
finer details of how the fMRI process works. 


Magnetic Fields and Radio Waves 


Our bodies are full of water (H20), especially for our brains. These hydrogen 
atoms play a crucial role in the function of the MRI. The magnet within an 
MRI has a strength of approximately ~1-3 tesla (T). In contrast, the earth’s 
magnetic field is only around 0.00005 T. This MRI's intense magnetic field, 
called BO, interacts with the protons within hydrogen atoms in the body. Most 
of these hydrogen atoms are found in water molecules in the body. The intense 
magnetic field aligns most protons parallel to this magnetic field along the 
longitudinal axis. Some higher energy state protons align antiparallel to BO 
[124]. 

When the protons align with the magnetic field, they process at the Larmor 
frequency rate. The Larmor frequency, or precession rate, is proportional to 
the strength, T, of the external magnetic field and is given by the equation: 


Wo = YBo (5.1) 


Where wo is the Larmor frequency, y is the gyromagnetic ratio (specific to the 
type of nucleus, i.e., hydrogen proton), and Bois the strength of the magnetic 
field [125]. 

Subsequently, a radiofrequency (RF) pulse is applied at a frequency equivalent 
to the Larmor frequency. This RF pulse provides electromagnetic energy to the 
protons, exciting them to a high-energy state. The net magnetization vector 
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aligns perpendicular to BO along the transverse plane when the protons are 
placed into this high-energy state. This process is commonly referred to as 
excitation. As the protons return to their low energy state and the net 
magnetization vector realigns with the magnetic field BO, the protons emit 
radio signals [126]. These radio signals are detected and used to form the 
images we see of the brain. 

This complex interaction between magnetic fields, radio waves, and the water 
within our bodies allows the MRI to produce detailed images of the body’s 
internal structures. 


T1 and T2 Relaxation Times 


There are two primary measurement parameters of the MRI: Tl and T2 
relaxation time constants. As discussed previously, the protons of mainly water 
molecules are aligned to the magnetic field and then energized into the 
transverse plane by the radiofrequency pulse. The T1 relaxation time constant 
measures the time it takes for protons to realign with the MRI’s magnetic field 
BO following the radiofrequency pulse. You may also see this as longitudinal 
relaxation or spin-lattice relaxation time. The T2 relaxation time constant is 
when the protons lose phase coherence with each other in the transverse 
plane[127]. This is commonly called spin-spin relaxation or transverse 
relaxation time. 

These two-time constants are important for the generation of the brain images 
that you see from an MRI. Images are commonly denoted as Tl or T2 
weighted, which will influence the contrast of various tissues in the brain. T1 
weighted images are useful for visualizing detailed brain anatomy. T2 
weighted images are useful for identifying pathological changes. 


Blood-Oxygen-Level Dependent (BOLD) Signal 


Now that we have a general understanding of how MRI functions, we can 
focus specifically on the application of fMRI. The key aspect of fMRI is the 
blood-oxygen-level dependent, or BOLD, signal. When neurons are active, 
they require and consume more oxygen. The body accomplishes this demand 
by increasing blood flow to the active neurons to deliver more oxygenated 
blood. Oxygenated blood has different magnetic properties from deoxygenated 
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blood. Specifically, oxygenated blood is diamagnetic, which means it repels 
magnetic fields, whereas deoxygenated blood is paramagnetic, which means it 
attracts magnetic fields. These differences, measured as the BOLD signal, 
produce a map of brain activity. 

As neuronal activity changes in a specific region, the proportion of oxygenated 
to deoxygenated blood changes, resulting in local measurable changes in the 
magnetic field. This resultant BOLD signal is technically an indirect measure 
of neuronal activity, reflecting the hemodynamic response to neural activation 
[123]. One advantage of the BOLD signal is the ability to provide high spatial 
resolution images in the 1-3 millimeters range. However, the BOLD response 
is delayed by several seconds due to the time it takes for the blood flow to 
increase in response to neural activity. This delay limits the temporal resolution 
of fMRI. 


Hardware Components of MRI 


There are three primary hardware components that you should familiarize 
yourself with: 

Magnets 

The primary magnet creates a strong magnetic field, BO. This magnet is 
typically a superconducting magnet cooled to very low temperatures using 
liquid helium. The strength of this magnet is measured in teslas (T), with 
clinical MRI machines typically operating at 1.5 —3 T. 

Gradient Coils 

These coils are responsible for the spatial encoding of the MR signal. They 
create small, linear variations in the magnetic field in the X, Y, and Z 
directions. This allows for the localization of the MR signal and the 
construction of images in different planes. 

RF Coils 


The RF coils transmit the RF pulses that excite the protons. The RF coils are 
also used to receive the signals emitted by the protons as they relax. 
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Figure 5.1: An MRI scanner [128] 


An MRI scanner is shown in Fig. 5.1 [128]. The large cylindrical structure 
contains the primary magnets that generate the BO field. The remaining 
hardware components are not visible in the image. 


Software for {MRI 


The processing and analysis of fMRI data is crucial for performing research 
and interpreting information. Two commonly used open-source software for 
pre-processing and post-processing fMRI data are FMRIB Software Library 
(FSL) and FreeSurfer [129], [130][7]. FSL is an open-source library of tools 
capable of performing in-depth analysis of fMRI data. It was created by “The 
Analysis Group” at the University of Oxford. FSL also offers tools for 
analyzing resting-state fMRI data, allowing researchers to study functional 
connectivity across different brain regions. Its user-friendly interface and 
comprehensive documentation make it accessible to researchers at all levels. 
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FreeSurfer is a similar software that the Laboratory developed for 
Computational Neuroimaging at the Athinoula A. Martinos Center for 
Biomedical Imaging. It is renowned for its capabilities in cortical surface 
reconstruction and volumetric segmentation. FreeSurfer's tools can accurately 
segment the brain into different regions of interest, providing detailed 
anatomical maps. Additionally, FreeSurfer excels in measuring cortical 
thickness and surface area, which are valuable metrics in studies of brain 
development and neurodegenerative diseases. 

These tools follow similar workflows for processing fMRI data. This includes 
preprocessing, registration (alignment), segmentation, statistical analysis, and 
visualization. These tools offer abundant guides and resources that are freely 
available online. By leveraging these tools, you can conduct unique 
experiments that contribute to our understanding of the brain’s workings. 


Experimental Design 


Designing an fMRI experiment involves careful planning to ensure that the 
data collected can answer the research question. Key elements include task 
design, control conditions, and timing. 


Task Design 

Task design is a critical element in {MRI experimental design, as it directly 
influences the brain activity that will be measured and analyzed. The tasks or 
stimuli presented to participants are carefully crafted to engage specific cog- 
nitive, sensory, or emotional processes, allowing researchers to investigate the 
neural underpinnings of these functions. 


Customization of Tasks to Research Questions 

The specific nature of the tasks used in an fMRI experiment highly depends 
on the research question being addressed. Researchers must design tasks that 
effectively target the brain regions or cognitive processes of interest. This cus- 
tomization ensures that the observed brain activity is directly related to the 
studied phenomenon. There are several different task types, such as: 
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Cognitive Tasks 

These tasks engage mental processes such as memory, attention, language, 
problem-solving, or decision-making. For instance, in a study investigating 
working memory, participants might be asked to keep a series of numbers or 
letters in mind and manipulate them mentally, such as by reversing their order. 
The brain activity during this task would primarily involve regions associated 
with working memory, like the prefrontal cortex. 

Language Processing Tasks 

To study language processing, tasks might involve reading sentences, listening 
to spoken words, or generating speech. For example, participants could be 
asked to read sentences that vary in syntactic complexity, allowing researchers 
to examine how different brain areas contribute to understanding and pro- 
cessing language. 

Decision-Making Tasks 

Decision-making tasks often involve presenting participants with choices that 
require evaluation and selection based on certain criteria, such as risk, reward, 
or moral judgment. These tasks can activate brain regions involved in execu- 
tive functions, such as the prefrontal cortex and the anterior cingulate cortex. 
Emotional Processing Tasks 

Emotional tasks are designed to elicit specific emotional responses. For exam- 
ple, participants might be shown images or videos that are sad, happy, or fright- 
ening. The resulting brain activity, particularly in regions like the amygdala 
and orbitofrontal cortex, can provide insights into how emotions are processed 
and regulated. 


Considerations in Task Complexity 

The complexity of the task is a crucial factor in fMRI experimental design. 
Tasks must be engaging enough to activate the brain regions of interest but not 
so difficult that they lead to participant frustration or disengagement, which 
could negatively impact the data quality. Researches should consider the fol- 
lowing when thinking about task complexity. 
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Task Engagement 

An engaging task ensures that participants are fully involved, which increases 
the likelihood of eliciting robust brain activity. For example, if the task is too 
simple, participants may not exert enough mental effort, resulting in minimal 
activation in the target brain areas. Conversely, if the task is overly complex, 
participants may become overwhelmed, leading to variable performance and 
potential artifacts in the data. 

Pilot Testing 

Before the actual fMRI study, tasks are often pilot-tested with a small group 
of participants. This helps fine-tune the difficulty level, ensuring it is appro- 
priate for the study population. Pilot testing can also identify task instructions 
or timing issues that could affect data quality. 

Task Timing and Duration 

The timing and duration of the task are also critical considerations. Tasks are 
typically organized into blocks or trials, with specific intervals between stimuli 
presentations to allow the brain to respond and recover. The duration of each 
task block or trial should be carefully calibrated to ensure that the fMRI scan- 
ner can reliably capture the brain activity. A duration that is too short may not 
allow for sufficient signal accumulation, while too long could lead to partici- 
pant fatigue. 


Example: Memory Recall Task 


Participants might be shown images or words during an initial encoding phase 
in a memory recall task. After a delay, they are asked to recall or recognize 
these items during retrieval. The brain activity during both the encoding and 
retrieval phases can be measured, allowing researchers to examine the neural 
mechanisms underlying memory formation and recall. During the encoding 
phase, participants are exposed to stimuli (e.g., images, words) and are in- 
structed to memorize them. This phase activates brain regions involved in 
memory formation, such as the hippocampus and the medial temporal lobe. In 
the retrieval phase, participants are asked to recall or recognize the previously 
presented stimuli. This phase engages regions involved in memory retrieval, 
such as the prefrontal cortex and parietal lobes. Comparing brain activity 
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between successful and unsuccessful recalls can provide insights into the pro- 
cesses that facilitate or hinder memory retrieval. 


Impact on Data Quality 

The task design directly impacts the quality of the data collected during an 
fMRI experiment. Poorly designed tasks can lead to uninterpretable or con- 
founded results. For example, if a task is too difficult, participants may fail to 
engage in the intended cognitive processes, resulting in weak or inconsistent 
brain activation patterns. Additionally, high variability in task performance 
across participants can introduce noise into the data, making it challenging to 
detect the true neural correlates of the task. 

In summary, task design is a fundamental aspect of fMRI experimental design 
that requires careful consideration of the research question, task complexity, 
and participant engagement. The task must be precisely tailored to elicit the 
desired brain activity while maintaining a difficulty appropriate for the study 
population. Proper task design, often refined through pilot testing, is essential 
for obtaining reliable and meaningful data in fMRI studies. 


Control Conditions 

Control conditions are essential in neuroimaging experiments as they allow 
researchers to differentiate the brain activity related to the experimental task 
from unrelated or background activities. The careful design of control condi- 
tions ensures that the observed changes in brain activity can be attributed to 
the specific cognitive or emotional processes being studied rather than to gen- 
eral factors such as attention, sensory processing, or motor responses. Differ- 
ent control conditions exist, such as resting-state scans and baseline tasks. 


Resting-State Scans 

In resting-state scans, participants are instructed to relax and avoid engaging 
in specific mental tasks. This condition is a baseline for measuring the brain's 
intrinsic activity without external stimuli or cognitive demands. Resting-state 
data can reveal the brain's default mode network, which is active when the 
brain is at rest and not focused on the outside world. Comparing task-related 
brain activity to this baseline helps isolate the neural responses specifically 
elicited by the experimental stimuli. 
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Baseline Tasks 


Baseline tasks are designed to be similar to the experimental task but lack the 
critical component under investigation. This design helps control for non-spe- 
cific factors such as visual or auditory processing, attention, or motor activity 
that might otherwise confound the results. For instance, in a study investigat- 
ing the brain's response to emotional stimuli, a baseline task might involve 
viewing neutral images similar in visual complexity to those used in the ex- 
perimental condition. This approach allows us to subtract the brain activity 
associated with general visual processing, leaving behind the activity specifi- 
cally related to emotional processing. 


Example: Emotional Processing Experiment 


Consider an experiment designed to investigate how the brain processes emo- 
tional stimuli. In the experimental condition, participants might be shown emo- 
tionally charged images, such as pictures of sad, happy, or fearful faces. These 
images will likely elicit strong emotional responses, engaging brain regions 
involved in emotion regulation, such as the amygdala, prefrontal cortex, and 
other limbic system areas. A carefully designed control condition is necessary 
to isolate the brain regions involved in emotional processing. In this case, the 
control condition might involve showing participants neutral images, such as 
pictures of ordinary objects or faces with neutral expressions. These images 
should be visually like the emotional stimuli in terms of complexity, bright- 
ness, and other visual features but should not evoke a strong emotional re- 
sponse. 

By comparing brain activity during the experimental condition (emotional im- 
ages) with the control condition (neutral images), researchers can identify the 
brain regions specifically activated by emotional processing. This comparison 
helps to eliminate the confounding effects of general visual processing, atten- 
tion, or other non-emotional factors, ensuring that the observed activity is truly 
related to the emotional content of the stimuli. 


Importance of Control Conditions in Data Interpretation 


Control conditions are crucial for accurate data interpretation in these experi- 
ments. Without appropriate control conditions, it would be challenging to 


214 


determine whether the observed brain activity is truly related to the cognitive 
or emotional processes of interest or simply a result of unrelated factors. 
Moreover, using well-designed control conditions allows for more precise lo- 
calization of brain function. By subtracting the activity observed during the 
control condition from that observed during the experimental condition, re- 
searchers can create a "difference map" highlighting the regions specifically 
involved in the task. This approach enhances data's spatial and functional res- 
olution, leading to more reliable and meaningful conclusions about brain func- 
tion. 

In summary, control conditions are an indispensable part of experimental de- 
sign in imaging studies. They allow researchers to isolate the brain activity 
associated with the specific cognitive or emotional processes under investiga- 
tion, ensuring that the results accurately reflect the neural mechanisms at play. 
Timing 

The timing of stimuli presentation and task execution in fMRI experiments is 
critical because the BOLD response, which fMRI measures, is not instantane- 
ous. The BOLD response reflects changes in blood flow and oxygenation lev- 
els in the brain that occur in response to neural activity. This response is de- 
layed, typically peaking about 5-6 seconds after the initial neural activity and 
slowly returning to baseline. Therefore, precise control over the timing of stim- 
uli and tasks is necessary to accurately correlate these with the BOLD signal 
and ensure that the collected data reflects the underlying neural processes. Dif- 
ferent types of experimental designs can be used to control the timing of stim- 
ulus, such as Event Related Design and Block Design. 


Event-Related Design 

An event-related design in fMRI experiments involves presenting individual 
stimuli or tasks at specific intervals, with each event being isolated in time. 
This design allows researchers to examine the brain's response to each stimulus 
separately, providing detailed information about transient neural processes. 
The event-related design offers high temporal resolution, allowing for the de- 
tection of neural responses to discrete events. Each stimulus is typically fol- 
lowed by a rest period or another stimulus, allowing researchers to observe 


215 


how the BOLD response evolves for each event. This design is also particu- 
larly useful for studying cognitive processes that occur over a short duration, 
such as decision-making, error processing, or rapid sensory perception. By an- 
alyzing the BOLD response to each event, researchers can investigate the tim- 
ing and sequence of brain activation across different regions. To prevent par- 
ticipants from anticipating stimuli, which could lead to confounding effects, 
the timing of stimuli presentation is often randomized or "jittered" (i-e., small 
variations in the timing of stimuli presentation are introduced). This approach 
helps to disentangle the neural responses to different stimuli and improves the 
statistical power of the analysis. 


Block Design 


In a block design, stimuli or tasks are presented in continuous blocks of time, 
alternated with rest or control blocks. This approach effectively detects sus- 
tained brain activity over time and is particularly useful for studying processes 
requiring sustained attention or involving continuous cognitive engagement. 
Block designs typically have a higher signal-to-noise ratio compared to event- 
related designs. This is because the sustained nature of the task during each 
block leads to a more pronounced and stable BOLD response. By averaging 
the BOLD response throughout the block, researchers can enhance the detec- 
tion of neural activity related to the task. Block designs are often used in stud- 
ies of sustained cognitive processes, such as language processing, motor tasks, 
or emotional regulation. For instance, in a study examining language pro- 
cessing, participants might be asked to read words aloud during the experi- 
mental blocks and view meaningless symbols during the control blocks. Re- 
searchers can identify the brain regions involved in language processing by 
comparing the BOLD responses during the experimental and control blocks. 
The alternating pattern of task and rest/control blocks allows researchers to 
compare brain activity across different conditions, such as experimental versus 
control or various types of stimuli. This comparison is crucial for isolating the 
specific neural correlates of the task or stimuli being studied. 
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Considerations for fMRI Experimental Design 


In addition to the choice of event-related or block design, several other factors 
must be carefully considered to ensure the success of an fMRI experiment: 


Duration of the Scan Session: 

The length of the scan session should be carefully planned to balance the need 
for sufficient data with participant comfort and attention. Scan sessions that 
are too long can lead to participant fatigue, boredom, or discomfort, which can 
introduce noise into the data. Shorter, well-structured sessions with adequate 
breaks can help maintain participant engagement and data quality. 

Number of Trials or Blocks: 

The number of trials (in an event-related design) or blocks (in a block design) 
should be sufficient to provide robust data for statistical analysis. A higher 
number of trials or blocks increases the reliability of the results by allowing 
for the averaging of multiple BOLD responses, which reduces the impact of 
random fluctuations in the data. However, this must be balanced against the 
potential for participant fatigue. 


Participant Instructions: 

Clear and concise instructions are essential to ensure participants understand 
and consistently perform the task throughout the scan. Inadequate or confusing 
instructions can lead to variability in task performance, which can confound 
the results. Practice sessions outside the scanner can be useful for familiarizing 
participants with the task before the scan. 

Minimizing Motion Artifacts: 

Motion artifacts are a significant concern in fMRI studies, as even small move- 
ments can lead to distortions in the BOLD signal. Participants should be in- 
structed to remain as still as possible during the scan to minimize motion. 
Comfortable padding and head restraints are often used to reduce involuntary 
movements. Additionally, real-time motion correction techniques can be ap- 
plied during data acquisition and analysis to further minimize the impact of 
motion artifacts. 
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Stimulus Presentation and Timing: 

The precise timing of stimulus presentation is crucial for correlating it with the 
BOLD response. This requires accurate synchronization between the fMRI 
scanner and the stimulus delivery system. Any delays or inaccuracies in timing 
can lead to misalignment between the task and the measured brain activity, 
compromising the validity of the results. 


Practical Example: Language Processing Study 


Let’s consider an example of a study on language processing. In such a study, 
participants might be asked to read words aloud during experimental blocks 
and view meaningless symbols during control blocks. The experiment could 
be designed as follows: 


Block Design 

Participants alternate between blocks of reading words (experimental condi- 
tion) and blocks of viewing symbols (control condition). Each block might last 
20-30 seconds, followed by a rest period. 

Task Timing 

The presentation of each word or symbol within the block is precisely timed, 
with a fixed interval between stimuli to allow for consistent BOLD responses. 


Control Blocks 

The control blocks with meaningless symbols help isolate the brain activity 
related to language processing, as the control condition engages similar sen- 
sory and motor processes without involving language comprehension. 
Analysis 

By comparing the BOLD response during the experimental and control blocks, 
researchers can identify brain regions, such as Broca’s and Wernicke’s areas, 
that are specifically activated during language processing. 

In summary, the timing and design of stimuli presentation and task execution 
in {MRI experiments are critical for accurately measuring the BOLD response 
and drawing meaningful conclusions about brain activity. Event-related and 
block designs each have their strengths and are chosen based on the nature of 
the cognitive processes being studied. Alongside these considerations, careful 
attention to scan duration, trial/block numbers, participant instructions, and 
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motion artifact reduction is essential for ensuring the quality and reliability of 
the data. 


Advantages and Disadvantages 


The main advantage behind fMRI use is that it is a non-invasive imaging 
technique. Additionally, fMRI is advantageous for its high spatial resolution, 
allowing for the precise localization of brain activity, typically 2-3 millimeters. 
This allows for mapping brain region activity when performing the specific 
functional tasks employed by the experimental design. 

A major disadvantage of the {MRI is that it has low temporal resolution. It does 
not effectively capture rapid changes in brain activity during tasks. The fMRI 
is also sensitive to artifacts, which can be introduced by motion and magnetic 
interference. 


Functional Near-Infrared Spectroscopy 
(fNIRS) 


The Basics of NIRS and fNIRS 


Functional Near-Infrared Spectroscopy (fNIRS) is another non-invasive 
imaging technique that measures brain activity by detecting changes in blood 
oxygenation. fNIRS uses near-infrared light to penetrate the skull and measure 
light absorption by oxygenated and deoxygenated hemoglobin. This allows 
researchers to infer brain activity based on changes in blood oxygen levels. 


Infrared Light 


Functional Near-Infrared Spectroscopy (fNIRS) utilizes near-infrared light, 
which can penetrate biological tissues and reach the brain's surface. This light 
typically ranges from 650 to 900 nanometers. The key to fNIRS is how this 
light interacts with hemoglobin in the blood. Hemoglobin exists in two forms: 
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Figure 5.2: An fNIRs device worn on the head and hardware components [131] 


oxygenated (HbO) and deoxygenated (HbR). These two forms absorb near- 
infrared light differently, providing a way to measure changes in blood 
oxygenation. 

Oxygenated hemoglobin primarily absorbs light at around 850 nanometers, 
while deoxygenated hemoglobin absorbs more light at around 760 nanometers. 
These differences in absorption arise from the molecular structure of 
hemoglobin, which changes depending on whether it is bound to oxygen. The 
absorption properties of HbO and HbR can be attributed to the iron-containing 
heme groups within the hemoglobin molecules. When hemoglobin binds to 
oxygen, the electronic structure of the heme group is altered, which in turn 
affects how the molecule interacts with light. This interaction results in 
different absorption patterns for HbO and HbR, making it possible to 
distinguish between them using near-infrared light [130]. 

Detectors placed on the scalp measure the intensity of the light that is either 
transmitted through or reflected from the brain tissue. By analyzing light 
absorption at the wavelengths corresponding to HbO and HbR, fNIRS can 
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determine the relative concentrations of these two forms of hemoglobin. The 
principles of fMRI hold for fNIRS, wherein brain regions that become more 
active consume more oxygen. 

The ability of near-infrared light to differentiate between oxygenated and 
deoxygenated hemoglobin forms the foundation of fNIRS. This non-invasive 
and portable technique provides valuable insights into brain function. 
Researchers can use fNIRS to study brain activity during various cognitive 
tasks, motor activities, and other functional processes in multiple 
environments and applications. 


Hardware Components of fNIRS 


The primary hardware components of an fNIRS system are shown in Fig. 5.2 
[131] and include: 
Light Sources 


Near-infrared light emitters that illuminate the scalp. 


Detectors 


Sensors that detect the light after it has passed through the brain tissue. 
Optodes 


Light sources and detectors are combined on the scalp to measure the 
hemodynamic response. 


Software for fNIRS 


Two common open-source software tools are used to process and analyze 
fNIRS data: Homer3 and NIRS-SPM[132], [133]. Both tools take advantage 
of the MATLAB platform. Homer3 is a software that the Boston University 
Neurophotonics Center developed. NIRS-SPM is a software package 
developed by the Bio-Imaging Signal Processing lab at KAIST in Korea. 
These tools allow for importing and pre-processing fNIRS data through 
filtering and motion correction. The statistical analysis performed by these 
software tools is commonly the General Linear Model (GLM), which 
determines the significance of observed changes in hemoglobin 
concentrations. Finally, these software provide their means for visualizing the 
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results. Like the fMRI software discussed, both tools have many resources 
available for free online. 


Experimental Design 
Designing an f{NIRS experiment involves: 


Task Design 


Like fMRI, participants perform tasks or are exposed to stimuli to elicit brain 
activity. These tasks are designed to engage cognitive, sensory, or motor 
functions, which in turn activate specific regions of the brain. The task choice 
depends on the research question and brain regions of interest. There are 
different types of tasks, such as cognitive (working memory and problem- 
solving tasks), motor (finger tapping and hand movements), or sensory (visual 
or auditory stimuli). For example, a working memory task would be employed 
in research to study prefrontal cortex activity. 


Placement of Optodes 


Careful placement of light sources and detectors on the scalp ensures accurate 
hemodynamic response measurement. Optodes are typically placed according 
to the 10-20 system (Chapter 4). This system helps ensure that optodes are 
positioned over specific cortical areas corresponding to the brain regions of 
interest. The distance between the source and detector optodes is typically 
between 2.5 and 4 cm. This distance is optimal for measuring the 
hemodynamic response from the brain's cortical layer. If the distance is too 
short, the signal may be dominated by superficial tissue, while a distance that 
is too long may result in signal loss or contamination by deeper structures. The 
configuration of optodes (the pattern in which they are placed on the scalp) is 
also important. The configuration can vary depending on the area of the brain 
being studied. For example, optodes might be arranged in a dense array over 
the forehead if the study focuses on the frontal cortex. The arrangement can 
also be multi-channel, covering large areas of the scalp for whole-brain 
measurement. 
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Control Conditions 

Used to isolate the brain activity related to the task from other sources of 
variability and are like that of fMRI. The unique aspects of fNIRS, including 
its sensitivity to changes in blood oxygenation and its non-invasive nature, 
make the design of control conditions, particularly important and include 
additional controls that researchers should consider, such as the following: 


Physiological Noise Control 

fNIRS is sensitive to physiological signals such as heart rate, respiration, and 
blood pressure, which can introduce noise into the data. Control conditions are 
designed to differentiate task-related brain activity from these physiological 
signals. For instance, during a resting baseline condition, participants might be 
asked to sit quietly without engaging in cognitive tasks. This allows 
researchers to measure the baseline hemodynamic response associated with 
resting physiology, which can then be subtracted from the task-related signals. 


Superficial Tissue Signal Isolation 

fNIRS measures changes in light absorption related to blood oxygenation in 
both the brain and superficial tissues like the scalp. Control conditions help 
isolate the signals originating from the brain by accounting for the superficial 
tissue responses. For example, in a task involving head movement, a control 
condition might involve the same movement without the cognitive task, 
allowing for identifying and subtracting movement-related artifacts. 


Ambient Light and Environmental Factors 

External factors such as ambient light or temperature changes can affect fNIRS 
measurements. Control conditions that match the environmental conditions of 
the experimental task but without cognitive or sensory stimulation help to 
identify and control for these external influences. This ensures that the 
observed brain activity is not confounded by environmental variability. 


Temporal Dynamics Control 

The hemodynamic response measured by fNIRS is inherently dynamic, with 
changes occurring over time as the brain processes stimuli. Control conditions 
that involve time-matched tasks without the specific cognitive demand help 
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distinguish between the time-related changes in brain activity due to the task 
and those that occur naturally or due to other factors. This can be particularly 
important in functions that involve learning or adaptation over time, where 
brain activity might change as the participant becomes more familiar with the 
task. 


Cognitive Load Control 

fNIRS is often used to study tasks involving varying cognitive load levels. 
Control conditions with different levels of cognitive load, such as a simple task 
compared to a more complex one, help isolate the specific brain regions and 
responses associated with increased cognitive demands. This approach allows 
researchers to understand better how cognitive load influences brain activity 
and to separate these effects from other factors. 


Advantages and Disadvantages 


Like fMRI, fNIRS is a highly advantageous non-invasive imaging technique. 
Its key strengths include portability and ease of use, which enable fNIRS ex- 
periments to be conducted in a wide range of environments—from traditional 
laboratory settings to more naturalistic or mobile conditions. This versatility 
makes fNIRS particularly valuable for studies that require flexibility and real- 
world applicability. Moreover, fNIRS offers excellent temporal resolution and 
can measure changes in brain activity in hundreds of milliseconds. This high 
temporal resolution allows researchers to capture rapid fluctuations in neural 
activity, which is essential for understanding dynamic brain processes and cog- 
nitive functions. 

Despite these advantages, fNIRS has notable limitations, primarily related to 
its spatial resolution and depth sensitivity. The spatial resolution of fNIRS is 
constrained to approximately 1-3 cm, which limits its ability to localize brain 
activity to smaller or closely spaced regions precisely. This lower spatial reso- 
lution means that fNIRS is less effective for tasks that require fine-grained 
anatomical detail. Additionally, the depth sensitivity of fNIRS is limited to 
around 1-3 cm beneath the cortical surface, which restricts its capacity to de- 
tect activity in deeper brain structures. This limitation poses challenges for 
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studying subcortical regions and understanding their roles in various neural 
processes. 

Efforts are underway to address some of these limitations, particularly by de- 
veloping multichannel NIRS systems. These advanced systems aim to improve 
spatial resolution by increasing the number of measurement channels and en- 
hancing the sensitivity to different depths. While these advancements hold 
promise for overcoming some of the current constraints of fNIRS, they are still 
in the developmental stages and have not yet fully realized the potential im- 
provements. Future enhancements in fNIRS may offer better spatial and depth 
resolution as research and technology evolve, expanding its applicability and 
effectiveness in neuroimaging studies. 


Comparing fMRI and fNIRS 


Both functional imaging techniques, {MRI and fNIRS, have distinct strengths 
and weaknesses that make them suitable for different research questions and 
settings. fMRI excels in providing high spatial resolution, allowing for detailed 
brain activity mapping with precise localization of neural processes. This high 
spatial resolution is particularly advantageous for studies that require an in- 
depth understanding of the anatomical specificity of brain functions and the 
accurate identification of neural networks involved in complex cognitive tasks. 
However, fMRI has limitations in temporal resolution, which means it is less 
effective at capturing the rapid fluctuations in brain activity over time. 
Additionally, fMRI is significantly more expensive and less portable than 
fNIRS, often requiring participants to remain still in a large, stationary 
machine, which can limit its applicability in more dynamic or naturalistic 
environments. 

On the other hand, fNIRS provides excellent temporal resolution, making it 
well-suited for capturing rapid changes in brain activity with a finer temporal 
granularity. This capability is crucial for understanding how brain activity 
evolves over short periods and studying processes that occur on a millisecond 
scale. Moreover, fNIRS is more portable and less expensive than fMRI, 
allowing for greater flexibility in experimental design and the possibility of 
studying participants in more naturalistic settings. This portability and ease of 
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use make fNIRS particularly valuable for research involving young children, 
individuals with movement disorders, or studies that require mobile or 
ecologically valid conditions. However, fNIRS has limitations in spatial 
resolution, which means it is less effective at pinpointing the exact locations 
of brain activity and is generally less capable of differentiating between closely 
spaced brain regions. 

Ultimately, the choice between fMRI and fNIRS depends on the specific 
research objectives and the nature of the experimental conditions. Researchers 
must weigh the importance of spatial versus temporal resolution and practical 
considerations such as cost and the study setting to determine the most 
appropriate imaging technique for their needs. 


Ethical Considerations 


Several considerations should be considered when conducting research using 
fMRI and fNIRS to ensure the study is conducted responsibly. Both fMRI and 
fNIRS involve collecting neuroimaging data, which can raise concerns about 
participants’ privacy and data security. Researchers must obtain informed 
consent that clearly explains the nature of the study, the procedures, and any 
potential risks or benefits. Additionally, researchers should ensure that 
participants have the autonomy and right to withdraw from the study at any 
time with no consequences. Data must be handled strictly to protect 
participants’ identities and personal information. The potential for unexpected 
results that could have clinical implications requires careful consideration, and 
participants should be informed about how these findings will be managed and 
communicated. Furthermore, researchers should be mindful of their findings' 
impact on individuals and groups and avoid potential data misuse that could 
lead to stigmatization or discrimination. Researchers should maintain 
transparency, respect participants’ rights, and ensure data integrity when 
conducting fMRI and fNIRS research. 


Chapter 5: Summary 


In conclusion, fMRI and fNIRS are powerful neuroimaging techniques that 
provide complementary insights into brain function. By understanding these 
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methods' principles, hardware, software, advantages, and disadvantages, 
researchers can effectively apply them in neuroengineering research. Each 
technique offers unique strengths—fMRI, with its high spatial resolution for 
detailed brain mapping, and fNIRS, which has excellent temporal resolution 
and portability for dynamic and naturalistic studies. Leveraging these strengths 
allows researchers to understand brain activity and its underlying mechanisms 
comprehensively. 

As you continue your exploration through this book and your research 
endeavors, I encourage you to apply these concepts and techniques to your 
experiments. Consider developing a novel fMRI experimental design to reveal 
new intricacies within the brain or analyze and interpret fNIRS data from 
previous research to uncover fresh insights. Given the knowledge and 
resources you now possess regarding fMRI and fNIRS, these challenges are 
well within your capabilities. Your ability to make meaningful contributions to 
the broader field of neuroengineering is within reach. 

The next adventure in your research journey will involve integrating what you 
have learned about neuroimaging techniques with exploring brain-computer 
interfaces (BCIs). BCIs represent a cutting-edge area of research that bridges 
the gap between neural activity and external devices, enabling direct 
communication between the brain and technology. In the upcoming chapters, 
you will build on your understanding of fMRI and fNIRS to delve into the 
design, implementation, and application of BCIs. This new domain will 
challenge you to apply your neuroimaging insights innovatively, potentially 
transforming how we interact with and understand the brain. Prepare to embark 
on this exciting frontier, where your skills and knowledge will contribute to 
pioneering advances in neuroengineering and beyond. 
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Chapter 5: Learning Activities 
Learning Activity 5.1 


Task: take 10 minutes to find online available images of 
fMRI 


Objective 
Students will search for and analyze functional Magnetic Resonance Imaging 
(fMRI) images to understand their applications and interpretations in 
neuroengineering. 


Materials 


e Internet access 
e Computers or tablets 
e Note-taking materials (paper, pens, or digital tools) 


Time 
10 minutes 


Instructions 


Step 1. Introduction (2 minutes): Start by explaining the purpose of the activity. 
Le., "Today, we are going to explore {MRI images. Functional MRI is a crucial 
tool in neuroengineering that helps us understand brain activity by measuring 
changes in blood flow." "Your task is to find online images of fMRI scans. You 
will have 10 minutes to find and download these images." 
Step 2. Searching for fMRI Images (6 minutes): Allow students to begin their 
search. 
Step 3. Sharing and Discussion (1 minute): Gather the students' attention once 
the time is up. 
Step 4. Analysis and Reflection (5 minutes): Facilitate a brief discussion about 
the images. Le. 
e "Who would like to share an image they found? Describe what you 
see in the fMRI scan and any interesting details." 
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e "What tasks or stimuli do you think caused the brain activity shown 
in the image?" 

e "How might these images be useful in neuroengineering research or 
clinical practice?" 


Step 5. Summarize the key takeaways from the activity. 

Follow-up Assignment: (Optional) Assign a follow-up activity where students 
write a short report on one of the {MRI images they found, including its source, 
the activated brain regions, and potential neuroengineering applications. 


Learning Activity 5.2 
Objective 


Students will engage in a problem-based learning activity to 


brainstorm and propose innovative solutions to reduce the 
cost of fMRI (functional Magnetic Resonance Imaging). 


Materials 
e Whiteboard or flip chart 
e = Markers 


e Internet access 

e Computers or tablets 

e Note-taking materials (paper, pens, or digital tools) 
Time 
30-45 minutes 
Instructions 


Introduction (5 minutes): 
1. Introduce the challenge and provide context about the high cost of 
fMRI and its importance in neuroengineering. 
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2. Explain the goal of brainstorming and proposing solutions to make 
fMRI more affordable. 


Group Formation (2 minutes): 


3. Divide the class into small groups of 3-4 students. 


Problem Exploration (5 minutes): 
4. Instruct each group to discuss the factors contributing to the high cost 
of fMRI, such as equipment, maintenance, personnel, and operational 
expenses. 


Research and Brainstorming (15 minutes): 
5. Guide the groups to research and brainstorm potential solutions to re- 
duce the cost. Encourage them to think creatively and consider: 
e Technological advancements 
e Alternative imaging methods 
e Cost-cutting measures 
e Innovative funding strategies 


Solution Development (10 minutes): 
6. Ask each group to choose and develop the most promising ideas into 
a coherent proposal. 
7. Have the groups prepare a brief presentation to share their solutions 
with the class. 


Presentation and Discussion (10 minutes): 
8. Facilitate the presentations, allowing each group 2 minutes to present 
their ideas. 
9. Lead a discussion on the feasibility and potential impact of the pro- 
posed solutions. 


Conclusion (3 minutes): 


10. Summarize the activity and highlight key takeaways, emphasizing the 
importance of innovative and collaborative problem-solving in neu- 
roengineering. 
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Learning Activity 5.3 

Activity: Buzz Groups and Pugh Chart 
Analysis - Comparing Brain Imaging 
Techniques 


Objective 


Students will collaborate in small (buzz) groups to compare different brain 
imaging techniques (({MRI, fNIRS, EEG, MEG) and create a Pugh chart to 
evaluate their advantages and disadvantages. 


Materials 


Whiteboard or flip chart 

Markers 

Internet access 

Computers or tablets 

Note-taking materials (paper, pens, or digital tools) 

Handouts with criteria for evaluation (e.g., cost, spatial resolution, 
temporal resolution, invasiveness, accessibility) 


Time 
45 minutes 


Instructions 


Introduction (5 minutes): 

1. Introduce the activity and explain the goal: to compare fMRI, fNIRS, 
EEG, and MEG using a Pugh chart. 

2. Briefly describe each brain imaging technique and the criteria for eval- 
uation (cost, spatial resolution, temporal resolution, invasiveness, ac- 
cessibility). 

Group Formation (2 minutes): 


3. Divide the class into small buzz groups of 3-4 students. 


231 


Research and Discussion (10 minutes): 
4. Assign each group one brain imaging technique (fMRI, fNIRS, EEG, 
MEG) to research and discuss. 
5. 5. Have each group gather information about their assigned technique, 
focusing on the evaluation criteria. 
Criteria Development (5 minutes): 
6. Come together as a class and agree on the specific criteria and their 
definitions for the Pugh chart. 
7. . Write the criteria on the whiteboard or flip chart for reference. 
Group Analysis and Pugh Chart Creation (15 minutes): 
8. Instruct each group to fill out the Pugh chart, evaluating their assigned 
technique against the criteria. 
9. Each group should provide a rating for their technique on each crite- 
rion (e.g., 1-5 scale) and justify their ratings. 
Presentation and Compilation (5 minutes): 
10. Have each group present their findings and ratings to the class. 
11. Compile the ratings into a single Pugh chart on the whiteboard or flip 
chart, discussing discrepancies and reaching a consensus. 
Conclusion (3 minutes): 
12. Summarize the activity and highlight the key takeaways, emphasizing 
the strengths and weaknesses of each brain imaging technique and the 
importance of such comparative analyses in neuroengineering. 


@ 


eee 
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Chapter 5: Lab introduction 


In this series of lab exercises, you will explore advanced 
neuroimaging data analysis techniques using FreeSurfer and 
MATLAB. These labs will provide hands-on experience with 
processing and analyzing fMRI data and practical guidance on using 
imaging tools for data analysis. 

You will start using FreeSurfer to process and analyze {MRI data 
from the OpenfMRI dataset ds000171. This dataset examines how 
neural processing of emotionally provocative auditory stimuli is 
altered in individuals with Major Depressive Disorder (MDD) 
compared to never-depressed control participants. Analyzing this 
data will give you insights into how emotional stimuli affect neural 
processing in different populations. 

Next, you will follow a YouTube tutorial by Mike X Cohen, which 
provides step-by-step guidance on analyzing MATLAB imaging 
data. This tutorial will help you apply MATLAB tools to real-world 
data analysis scenarios, enhancing your skills in neuroimaging 
analysis. If you have not yet installed MATLAB, instructions for 


downloading it are available in Chapter 1, Lab Example 1. These 


labs will prepare you for further neuroimaging research and data 
analysis exploration. 
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Chapter 5: Lab Example 1 


Overview 


For this laboratory exercise, we will utilize FreeSurfer to 
process and analyze fMRI data from the OpenfMRI dataset 
ds000171. This dataset examines how neural processing of 


emotionally provocative auditory stimuli is altered in depression. The study 
includes 19 individuals with Major Depressive Disorder (MDD) and 20 never- 
depressed control participants (ND) who listened to positive and negative 
emotional, musical, and nonmusical stimuli during fMRI scanning. 


Dataset Description 
e Title: Neural Processing of Emotional Musical and Nonmusical 
Stimuli in Depression 
e =©Participants: 19 MDD and 20 ND 
e Tasks: Music comprehension/production 
e Scanner: Siemens Skyra 3T 


Objective 


This laboratory exercise aims to process, visualize, and compare the brain 
activity of control and MDD subjects using FreeSurfer utilizing the dataset 
described above. This exercise will guide you through downloading the 
dataset, setting up FreeSurfer, running preprocessing with FreeSurfer, and 
visualizing the data in Freeview (the FreeSurfer visualization interface). 


Installation and Setup 


Now that you know about this laboratory exercise, let’s get FreeSurfer installed 
and running. As of the time of writing this book, FreeSurfer 7.4.1 is the current 
version of the toolbox; we will be covering the basic installation via a 
Windows 11 64-bit OS, running on an LG Gram laptop with 16GB of RAM 
and an integrated graphics card. FreeSurfer is intended for use on a Linux 
platform. Installation with Windows is less straight-forward. Following the 
FreeSurfer installation tutorial page is recommended, but this is meant to serve 
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as a brief guide. First, fill out this information to obtain a license key: 
https://surfer.nmr.mgh.harvard.edu/registration.html. 


Now, let’s begin by opening Windows PowerShell as an administrator. [Hint: 
you can hit winkey+x, then select Terminal (admin)]. We may generally refer 
to this environment as a “terminal” from here on out. From there, type the 
following command: 


wsl --install 


This command will install Windows Subsystem for Linux (WSL). You will see 
a progression bar followed by a successful install. You will likely be prompted 
to generate a username and password. Using PowerShell, you will notice 
slightly different colors within the terminal window, indicating you are in a 
different environment. 

Download an X server to handle the graphical visualizations. You can search 
for x servers and follow the installation instructions or install one of the 
recommended options from FreeSurfer; this includes Xming and VcXsrv. The 
installation process here is straightforward: follow the default options and 
proceed. 

Next, let’s return to the Ubuntu terminal. If you closed out of it or are returning, 
re-open PowerShell and type “ws/” (enter), then type “cd” (enter). You are 
now in the Ubuntu environment's home directory. Finally, download 
FreeSurfer with the following* (See note below): 


wget 
https://surfer.nmr.mgh.harvard.edu/pub/dist/freesurfer/7 
.4.1/freesurfer-linux-ubuntu22_amd64-7.4.1.tar.gz 


*Note: replace the link following “wget” with the latest release found here: 
https://surfernmr.mgh.harvard.edu/fswiki/rel7downloads. Copy the link for 


the relevant Linux environment you're working in. I am installing the Ubuntu 
22 amd64 tar archive link. 
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This will take a while to download. While you’re patiently waiting for this, 
now would be a good time to go back and read the learning outcomes for this 
chapter. Can you respond to the learning outcomes aloud in your own words? 
If not, that’s okay; you’ve got plenty of time to master the concepts during 
your engineering journey. 

Welcome back! Let’s ensure the file was downloaded by typing “/s”’ (enter). 
You should see the file name listed. Let’s organize our work by creating a 
directory to unpack and install this file. 


sudo mkdir /usr/local/ 


This folder should already exist. If you installed the tar.gz file type, now type: 


sudo tar -xzvf freesurfer-linux-ubuntu22_amd64- 
7.4.1. tar.gz -C /usr/local 


You’re almost there! Now, enter the following commands to setup 
environmental variables, making it easier to navigate within the terminal: 


export FREESURFER_HOME=/usr/local/freesurfer/ 
echo “export FREESURFER_HOME=/usr/local/freesurfer/”> 
~/.bashrc 


Typing “ls $FREESURFER HOME” should list several folders of the 
FreeSurfer install. You were told to complete the registration information and 
obtain a license.txt file earlier. Download it and copy it in your Linux directory 
using: 


cp /mnt/c/Users/bfugg/Downloads/license.txt ~/ 


Typing “ls” should show the license.txt file listed. 
Next, enter the following: 


echo "export FS_LICENSE=$HOME/license.txt" >> ~/.bashrc 
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Configure display settings: 


echo “export DISPLAY=:@" >> ~/.bashrc 


Figure 5.3: Display Settings 
source ~/.bashrc 


Run Xlaunch, which we installed earlier, then try to run Freeview in the 


terminal with: 
freeview 


If the window opened, as in Fig. 5.3, great job. Let’s proceed to download the 
dataset we want to analyze. You can download or use any data that suites your 


experiment goals. 
Visit the OpenfMRI data page and download the ds000171 accession number 


dataset. 
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https://openfmri.org/dataset/ds000171/ 


Extract the folders you downloaded into your WSL home directory: 


mkdir -p ~/openfmri_ds@@0@171/controls 

mkdir -p ~/openfmri_ds@@@171/mdd 

tar -xzvf controls.tar.gz -C 
~/openfmri_ds@@Q@171/controls 

tar -xzvf mdd.tar.gz -C ~/openfmri_dseee171/mdd 


Preprocessing 


If you explore the data structure within the controls and MDD folders, you’ ll 
notice several subfolders separating each subject data and anatomical and 
functional fMRI data. Let’s work with subject 01 data from the control and 
MDD groups. Setup a SUBJECTS _DIR environment variable: 


export SUBJECTS DIR=~/openfmri_ds@@@171 


Run “recon-all” for the control subject and then the MDD subject; this might 
take a long time to process. 


recon-all -s control _sub-@1 -i 
~/openfmri_ds@@Q@171/controls/sub-control@1/anat/sub- 
control@1_T1iw.nii.gz -all 

recon-all -s mdd_sub-@1 -i ~/openfmri_ds@00171/mdd/sub- 
mdd@1/anat/sub-mdd@1_T1w.nii.gz -all 


Run ‘qcache’ for both subjects to smooth the data. 


recon-all -s control _sub-@1 -qcache 
recon-all -s mdd_sub-@1 -qcache 


The ‘recon-all’ command in FreeSurfer is a preprocessing tool that generates 


detailed 3D models of the brain's structure, essential for accurate registration 
and analysis of functional MRI data in subsequent steps. 
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Now, you must run FS-FAST (FreeSurfer Functional Analysis Stream) 
preprocessing. FSFAST performs essential preprocessing steps for functional 
data, such as motion correction, slice timing, spatial normalization, and 
smoothing. First, ensure your data is organized into the hierarchy FSFAST 
expects. This requires your data to be organized into run directories (001,002, 
etc) within a ‘bold’ subfolder for each subject. This can be accomplished by 
the following: 


mkdir -p ~/openfmri_ds@@0171/analysis/control_sub- 
@1/bo1d/ee1 

mkdir -p ~/openfmri_ds@@0171/analysis/control_sub- 
@1/bo1d/ee2 

mkdir -p ~/openfmri_ds@0@0171/analysis/mdd_sub- 
@1/bo1d/ee1 

mkdir -p ~/openfmri_ds@0@0171/analysis/mdd_sub- 
@1/bo1d/ee2 


Now move and rename the functional data files into these directories: 


mv ~/openfmri_ds@@0171/controls/sub-control@1/func/sub- 
control@1_task-music_run-1_bold.nii.gz 
~/openfmri_ds@@@171/analysis/control_sub- 
01/bold/@01/Ff.nii.gz 

mv ~/openfmri_ds@@0171/controls/sub-control@1/func/sub- 
control@1_task-music_run-1_events.par 
~/openfmri_ds@@@171/analysis/control_sub-@1/bold/ee1/ 
mv ~/openfmri_ds@@0171/controls/sub-control@1/func/sub- 
control@1_task-nonmusic_run-4_bold.nii.gz 
~/openfmri_ds@@Q@171/analysis/control_sub- 
01/bold/@02/Ff.nii.gz 

mv ~/openfmri_ds@@0171/controls/sub-control@1/func/sub- 
control@1_task-nonmusic_run-4 _events.par 
~/openfmri_ds@@Q@171/analysis/control_sub-@1/bold/ee2/ 
mv ~/openfmri_ds@00171/mdd/sub-mdd@1/func/sub- 
mdd@1_task-music_run-1_bold.nii.gz 
~/openfmri_ds@@Q@171/analysis/mdd_sub- 

@1/bold/001/f .nii.gz 
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mv ~/openfmri_ds@00171/mdd/sub-mdd@1/func/sub- 
mdd@1_task-music_run-1_events.par 
~/openfmri_ds@@Q@171/analysis/mdd_sub-@1/bold/ee1/ 
mv ~/openfmri_ds@00171/mdd/sub-mdd@1/func/sub- 
mdd@1_task-nonmusic_run-4_bold.nii.gz 
~/openfmri_ds@@Q@171/analysis/mdd_sub- 
01/bold/@02/f.nii.gz 

mv ~/openfmri_ds@00171/mdd/sub-mdd@1/func/sub- 
mdd@1_task-nonmusic_run-4_events.par 
~/openfmri_ds@@Q@171/analysis/mdd_sub-@1/bold/ee2/ 


To reiterate, for this example, we are only working with a few of the run files 
for this walk-through, but you should organize your data into this structure for 
all your data if you plan to conduct group-level analysis. Create a text file in 
each session folder containing the subject name that corresponds to the 
anatomical data processed by recon-all earlier: 


echo "control _sub-@1" > 
~/openfmri_ds@@Q@171/analysis/control_sub-@1/subjectname 
echo "mdd_sub-@1" > 
~/openfmri_ds@@Q@171/analysis/mdd_sub-@1/subjectname 


Now we are ready to run FS-FAST preprocessing: 


export SUBJECTS DIR=~/openfmri_ds@@Q@171/analysis 
preproc-sess -s control _sub-@1 -fsd bold -stc up - 
surface fsaverage lhrh -mni3@5 -fwhm 5 -per-run 
preproc-sess -s mdd sub-@1 -fsd bold -stc up -surface 
fsaverage lhrh -mni305 -fwhm 5 -per-run 


After completing this, you can check the registration quality using the 
following codes. A value closer to 0 indicates good registration: 


tkregister-sess -s mdd_sub-@1 -fsd bold -per-run -bbr- 
sum 
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Next, we need to analyze the data, which is called first-level analysis. For 
simplicity, we will only analyze one run for two subjects, one control, and one 
MDD subject. However, this will not yield results that can be interpreted to 
any degree of certainty. Realistically, you will perform the analysis on the 
entire data set to increase the statistical robustness of your analysis and form 
more reliable conclusions. Let’s review the data we are working with and set 
up our analysis, create contrasts, and run the analysis. 

Recall that the dataset we are working with analyzed Major Depressive 
Disorder (MDD) subjects and never-depressed control participants (ND), who 
listened to positive and negative emotional musical and nonmusical stimuli 
during fMRI scanning. The participants would listen to a tone (null-condition), 
then listen to either positive or negative music, then respond to the researchers 
if they thought the music was positive or negative. From this, we observe three 
non-null conditions: response, positive music, and negative music. As we 
proceed, we may decide to compare the brain response during music vs. 
baseline (tone) or positive music response vs negative music response, etc., 


where: 
0 = tone 
1 = response 


2 = positive music 
3 = negative music 


Begin with setting up the analysis: 


mkanalysis-sess \ 
-fsd bold -stc up -surface fsaverage lh -fwhm 5 \ 
-event-related -paradigm sub-control@1_task-music_run- 
1_events.par -nconditions 3 \ 
-spmhrf @ -TR 3 -refeventdur 16 -nskip 4 -polyfit 2 \ 
-analysis control.music.sm@5.lh -force -per-run 


This sets up our analysis for just the left hemisphere (lh); you can do this for 
both the right hemisphere (rh) and subcortical structures (mni305). We are just 
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going to do lh for this exercise. Run the same analysis setup for the mdd 
subject. 

Next, create contrasts you are interested in examining. For example, the 
following would contrast positive and negative music vs. baseline, positive 
music vs baseline, and response vs baseline. 


mkcontrast-sess -s control _sub-@1 -analysis 
control.music.sm@5.1lh -contrast combined_music-vs-rest - 
a2 -a 3 

mkcontrast-sess -s control _sub-@1 -analysis 
control.music.sm@5.1h -contrast pos_music-vs-rest -a 2 
mkcontrast-sess -s control _sub-@1 -analysis 
control.music.sm@5.1h -contrast response-vs-rest -a 1 


Again, you can use any contrasts or nomenclature you’d like. For example, the 
values -a 2 -a 3 indicate you want conditions 2 and 3 active. You should create 
contrasts for the mdd subject as well. Finally, run the analysis with: 


selxavg3-sess -s control _sub-@1 -analysis 
control.music.sm@5.1h 
selxavg3-sess -s mdd_sub-@1 -analysis mdd.music.sm@5.1h 


The above commands are just for the left hemisphere; you would repeat each 
command for rh and mni305. 

Suppose you have made it this far. Congratulations! You have completed a 
first-level analysis of one subject (Fig. 5.4). You can now visualize your data. 
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Figure 5.4: Set up for analysis 


Visualization and Comparison 


We can start by viewing the basic anatomical data (Fig. 5.5). Open freeview 
with your terminal's “freeview” command. Select load volume and locate the 
“mri” directory of the subject you wish to look at. Alternatively, you can look 


Figure 5.5: Basic Anatomical Data 
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at other interesting outputs from recon-all, such as the segmentations, surfaces, 
or masks. Below are some examples of the view you can load that may assist 
you in your research goals. Many of these can be generated by utilizing the 
Freeview GUI screen and loading the volumes or surfaces you wish to 
investigate. Adjusting the opacity and overlays on the left side of the GUI 
provides additional ways to customize the viewport. 

If you followed the directions above, we analyzed three non-null conditions 
during the analysis. 

The inflated left-hemisphere view shows combined music vs rest in control 
subjects to the right. The red and yellow highlights show significant activations 
of the superior temporal gyrus. Red indicates a statistically significant 
difference in activation during music vs baseline (tones). 

For the major depressive disorder subject, let’s look at the results for their 
combined music vs baseline (tone). You can see similar areas of activation in 
the left hemisphere. If you analyzed the right hemisphere and the subcortical 
structures, you could examine which internal structures become activated in 
each subject, as shown below. 

The applications of this tool are limitless in your journey to studying fMRI. 
From here, you could analyze every control subject and create a statistically 
robust representation of brain activations during music exposure compared to 


Figure 5.6: Combined music vs rest in control subjects 
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the resting state. Or you could proceed to compare the activations of the control 
subject to the depressive subjects. 


Figure 5.7: Combined music vs rest in major depressive disorder subjects 


A 


245 


Chapter 5: Lab Example 2 


Overview 


In this lab example, you will follow a YouTube tutorial by 
Mike X Cohen [1]. Mike X Cohen has many tutorials on his 
YouTube platform, where he guides viewers through many 


examples of imaging data analysis using engineering tools. For this lab, we 
will be using MATLAB as our tool. You should already have MATLAB 
downloaded; if you do not, instructions on downloading MATLAB can be 
found in Chapter 1, Lab Example 1. 

To get started, here is a link to Mike X Cohen’s YouTube page: 


https://www.youtube.com/@mikexcohen1 


Once you are on his page, you want to navigate to the Playlists tab and then 
find the Essentials of Neuroscience with MATLAB playlist. When the playlist 
is open, you will notice on the right-hand side that there are several modules 
in this playlist. For this lab example, we will follow Module 4 (FMRI). In this 
module, Mike X Cohen provides an example of processing fMRI data from an 
experiment using visual stimuli. As mentioned in Chapter 8, the brain's visual 
cortex is retinotopically organized, and in this lab, we will be able to see that 
organization. Mike X Cohen uses a data set from Kay et al., 2020 [134]. In this 
study, the researchers are testing a new method called Temporal 
Decomposition to improve the spatial resolution of {MRI data analyzed with 
tasked tasks. 


Video 1 


In Module 4-1, Mike X Cohen explains the task in this study. In the study, 
participants looked at visual stimuli, where they needed to look at the dot in 
the center of a ring. The ring’s eccentricity changed in size throughout six 
conditions, where in condition one, the ring was closest to the dot, and in 
condition six, it was furthest away from the dot. 

This task presents visual stimuli from different distances from the fovea. Due 
to the retinotopic organization of the visual cortex, the visual stimuli produced 
by condition one, where the ring is closest to the center dot, activate central 
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vision in the fovea. Higher visual cortex activation will occur at the calcarine 
fissure (refer to Chapter 8 Fig. 8.6 [135]). As the visual gets further from the 
center, the stimuli will progressively move away from the calcarine fissure. 
You will see this progression of activation in the images generated by the lab. 
After you have found Mike X Cohen’s page and this module, go to the caption 
of the Module 4-1 video. In the caption, Mike X Cohen has provided all the 
code and data files needed for this lab. Once you click on the link, a file will 
download. In this folder, you will see several MATLAB files. You will first 
want to open the exampledataset.mat file and an Import Wizard box will pop 
up. 

You will want to click finished and have the data imported into your workspace 
in MATLAB. 


“a 


Select variables to import using checkboxes 
© Create variables matching preview. 
Create vectors from each column using column names. 


Create vectors from each row using row names. 


Variables in C:\Users\merce\AppData\Local\ Temp\f4caa457-4a83-4bae-9a2b-71acf978a627_matlab4neuros_module4 (1).zip.6... 


Import Name Size Bytes Class | |No variable selected for preview. 

@ Hbc 121695x1 973560 doubl{ 
@  [ildata 1x4 716540576 cell 

i (QO) design — 1x4 5248 cell 

i vj extrap... 1185x859 1017915 logica 
i] Ed imgloo... 1185x859 4071660 uint32) 
iv] to stimdur = 1x1 8 doubl 
iv] HH tr 1x1 8 doubl 

Help < Back Next (_) Generate MATLAB code Cancel 


Figure 5.8: Inserting example data 


Once you have finished this, you will want to navigate to 
matlab4neuros_module4 _partial.mat file. This file contains the code Mike X 
Cohen will walk you through in each video, where you can code with him to 
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produce the figures. It is recommended that you use this code in a live script. 
When the code is uploaded into MATLAB, you will be prompted to “Open this 
file as a live script.” You will want to click “Open as a live script.” 

You will then watch Mike X Cohen’s videos for Module 4 (FMRI). This lab 


Figure 5.9: Live Script View 


has several parts, during which you will create flat maps of the fMRI data, 
different visualizations for the BOLD response, and conduct a t-test. 


Video 2 


In Video 2, you can create and visualize the flat maps. A flat map is a two- 
dimensional representation of the brain’s cortical surface. Flat maps are made 
by reconstructing the cortical surface and then inflating it to smooth the gyri 
and sulci. Then, the surface is flattened into a 2D map, preserving the spatial 
relationships of the cortical regions. Here is a video from YouTube from MNE- 
Python that shows how flat maps are created. 


https://www.youtube.com/watch? v=OOy7tl yq8IM 
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You can see the flat maps by the end of Module 4, Video 2 of Mike X Cohen’s 
tutorial (Fig. 5.10). 
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Figure 5.10: Visualizing flat maps 
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Figure 5.11: Bold Time Course 
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Video 3 


After you have completed Video 2, you will move on to Video 3, where you 
will preprocess the BOLD signal data. In this video, you will produce a figure 
showing the BOLD time course from a specific voxel (Fig. 5.11). In the figure, 
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Figure 5.12: Design Matrix over time 


the x-axis represents the time in seconds (s), and the y-axis represents the 
intensity of the BOLD signal in arbitrary units (a.u.). This figure depicts 
changes in the blood oxygenation of this voxel over time. 


Video 4 


Once you finish with Video 3, you will move onto Video 4, exploring the 
design matrix the researchers used in their experimentation. In this video, you 
will produce three different visualizations. 

The first figure you create depicts when they presented each stimulus category 
over time (Fig. 5.12). 

Can you say that it is quite hard to understand the design matrix when it is in 
this form? Mike X Cohen explains this in this video and shows you how to 
convert this into a gray figure using the following code. 


% perhaps it's easier to visualize as an image? 
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imagesc(design{1}) 
xlabel('Condition number' ) 
ylabel('Time (TR)') 
colormap gray 


Your figure will now look at Fig. 5.13. Now, it is much easier to see which 
condition (x-axis) has been presented over time (y-axis). The next step you 
will take in this video is to plot the events over the time course of the BOLD 
response. You will produce a figure that looks like Fig. 5.14. 


Condition number 


Figure 5.13: Gray Colormap of Design Matrix 


Next, you will produce Fig. 5.15, visualizing the even related data matrix. It 
will look like this. This figure shows the differences in BOLD activity over 
time (x-axis) for the different voxel indices (y-axis). This matrix allows for 
visualizing how the BOLD signal varies across the different voxels during a 


task over time. 
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Video 5 
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Figure 5.14: Events over time with BOLD response 
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Figure 5.15: BOLD signal across voxels during task 
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Figure 5.16: Gaussian Heat Map 


Next, you will watch Module 4, Video 5 of Mike X Cohen's tutorial, where 
you will create an animation of the bold response over time. You will first 
make a 2D Gaussian distribution heal mat map (Fig. 5.16). In a Gaussian dis- 
tribution, values are highest in the center and taper off symmetrically toward 
the edges. 

Next, you will produce the animation, in which you will see the progression of 
the BOLD response over time for each condition of the visual task from the 
study. 


Video 6 


Next, you will watch Module 4, Video 6 of the tutorial, where Mike X Cohen 
will set you up for the statistical analysis of the data, and you will create a new 
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data visualization. Following this tutorial, we encountered an error message 
stating, “Dot indexing is not supported for variables of this type.” This error 
was fixed by coding in 


datacursormode on 


Under % step 1: turn datacursormode on and click on a map 
instead of put it in the command window. 

Your figure will look at Fig. 5.17. 

Here, you see the BOLD response (y-axis) over time (x-axis) for the different 
conditions for the selected voxel. 


Video 7 


In the final video, Module 4, Video 7 of the Mike X Cohen tutorial series, you 
will conduct a t-test and visualize it on a map. A t-test is a statistical test that 
compares the means of two groups to determine if they are significantly 
different from each other. In this case, you will compare the differences in 


BOLD response 


Figure 5.17: BOLD response over time 
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BOLD activity for condition one, where the ring is closest to the center dot, 
and condition six, where the ring is the furthest from the center dot. The figure 
you will generate looks like this. 

In Fig. 5.17, you will see red and blue colors. Red represents areas where 
condition one has a significantly higher BOLD response than condition six. 
This makes sense since the red coloring is located at the calcarine fissure. Blue 
represents areas where condition six has significantly higher BOLD responses 
than condition one. This makes sense since blue is farther away from the 
calcarine fissure, indicating visual stimuli in peripheral vision. It is worth 
noting that Mike X Cohen explains why a t-test is not the best way to analyze 
this data. This is because there are a lot of variances in the BOLD response 
that will be ignored when conducting a t-test at a certain time. However, 


T-test on conditions 1 vs. 6 
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Figure 5.18: T-test map 
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learning how to conduct t-tests in MATLAB is good since they are 
fundamental statistical tests. 

That will conclude the {MRI tutorial from Mike X Cohen on YouTube. Mike 
X Cohen provides several other examples of understanding data from imaging 
techniques in different tutorials on his YouTube page, which was linked at the 
beginning of this lab. 

YouTube is a tool that you can use to find a lot of tutorials using engineering 
tools to get started learning the platforms. 
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Chapter 6 


Bridging Minds and Machines: 
Brain-Computer Interfaces 
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Introduction and Learning Objectives 


So far in this textbook, you have learned about the fundamental building 
blocks of the brain and different ways that researchers use to try and 
understand the brain, whether that be computational, such as in Chapters 2 and 
3, or through engineering tools, such as those discussed in Chapters 4 and 5. 
In this next chapter, we will explore Brain-Computer Interfaces (BCIs) and 
how researchers leverage our understanding of the brain to develop 
technologies that enable direct communication between the mind and 
machines. We will discuss the history of BCIs, their work, ethical 
considerations, and future implications of these technologies. By the end of 
this chapter, you will be able to: 


1. Identify and explain the critical hardware and software components 
involved in BCIs, including capturing brain signals, signal 
processing, and output devices. 


Discuss the advantages and disadvantages of comparing invasive 
and noninvasive BCIs. 

Grasp the ethical considerations and potential societal impacts of 
BCI technologies. 


Introduction to Brain-Computer Interfaces 
(BCIs) 


Definition of BCTs 


Brain-computer interfaces (BCIs) allow communication between the brain and 
external devices, offering individuals new ways to interact with their 
environment. By capturing and interpreting brain signals, BCIs can empower 
individuals to control computers or operate prosthetic limbs using only their 
thoughts. This technology can significantly improve the quality of life for 
those facing severe physical disabilities, helping them regain lost functions. 
BCIs achieve this by creating an interface that can translate neural activity into 
commands. This process begins with detecting brain signals through various 
methods such as EEG or MEG (Chapter 4), {MRI (Chapter 5), or invasive 
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approaches involving implanted electrodes, which we will discuss in more 
detail later in this chapter. The detected signals are amplified and processed to 
filter noise and extract features. Advanced algorithms decode these features 
and translate them into commands to control external devices, whether 
computers or prosthetics [136]. 

The applications of BCIs can be vast. In the medical field, BCIs can be used 
to restore communication and mobility for patients who suffer from disabilities 
caused by disease, injury, or stroke. For example, a person who suffers a spinal 
cord injury and has lost the ability to move their limbs can use BCIs to operate 
motorized wheelchairs or robotic prosthetics to regain levels of independence 
[137]. Beyond medical applications, BCIs are also being researched to enhance 
human-computer interaction, such as gaming and virtual reality experiences 
(Chapter 10). The creation and advancement of BCIs have opened a new 
frontier in how we interact with technology. 


Brief history and evolution of BCTs 


Dr. Grey Walter was one of the earliest researchers to develop BCIs (Fig. 6.1 
[138]). In the 1960s, he demonstrated the potential for using brain signals to 
control external devices [139]. His work laid a foundation for the future of 
direct mind-to-machine communication. In the 1970s, Dr. Eberhard Fetz made 
significant 

contributions to the 
field by showing 
that monkeys can 
be trained to 
control the firing 
rates of neurons in 
the motor cortex. 
He proved _ that 
neural activity can 


be harnessed to 
control external 


devices [140]. Figure 6.1: Dr. Grey Walter and Vivian Dovey 1943 [138] 
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These pioneers in the field of BCIs set the stage for the creation of more 
sophisticated BCI systems. 

BCIs gained momentum in the 1990s with the advancement of computer 
technology and neuroimaging techniques fueling this growth. These 
advancements allowed for a more precise and detailed interpretation of neural 
activity. Researchers commonly used EEGs to capture brain signals in these 
early stages of BCI development, and these early noninvasive BCIs were 
crucial in demonstrating the feasibility of brain-controlled communication and 
control. In 1998, there was a significant breakthrough in BCI development 
when Dr. Phillip Kennedy implanted the first intracortical BCI in a patient. 
This invasive BCI allowed a paralyzed individual to control a computer cursor 
by recording motor cortex activity [141]. This milestone showed the potential 
for invasive BCIs for individuals with severe motor impairments. 

In the early 2000s, this field experienced significant growth due to 
improvements in signal processing algorithms and machine learning 
techniques. These systems could not accurately decode the brain signals and 
were more applicable to real-world applications. During this time, researchers 
focused on extending these technologies outside the lab into a clinical setting 
to improve the quality of life of those with disabilities. The 2010s brought big 
companies such as Neuralink and BrainGate into the BCI field, where these 
companies wanted to develop more precise and varied controlled BCIs. 
Founded by Elon Musk, Neuralink aims to create BCIs that seamlessly 
integrate with the human brain, with the potential of a wide range of 
applications from medical rehabilitation and cognitive enhancement. 
BrainGate has shown significant progress in developing BCIs that allow 
individuals with tetraplegia paralysis due to injury of the cervical spinal cord, 
where individuals experience motor loss of all four limbs, to control robotic 
limbs and computer interfaces [137]. 

The history and milestones of BCIs relate to a rapidly advancing field that is 
increasingly integrating into medical care and daily life. As researchers 
continue to push the boundaries in this field, BCIs hold promise in 
transforming the lives of individuals with disabilities, enhancing human 
capabilities, and enhancing our interaction with technology. 
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Components of BCIs 


Now that we have discussed the foundations of BCIs and a general idea of 
their function, we will discuss how BCIs can bridge mind and machine in 
deeper detail. 


Capturing Brain Signals (EEG, {MRD 


Previously in this book, you explored techniques that capture electrical 
(Chapter 4 EEG/MEG) and hemodynamic signals (Chapter 5 fMRI and fNIR). 
These signals can also be captured for capturing brain signals for BCIs. For a 
quick recall, EEGs record electrical activity along the scalp by dating voltage 
fluctuations resulting from ionic current flows within the neurons. fMRIs 
measure brain activity based on blood flow; when a brain area is in use, we 
expect an increase in blood flow, allowing researchers to understand brain 
active regions. We will focus on how these techniques are used in BCIs. 
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Figure 6.2: EEG based BCI for capturing electrical activity [145] 
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EEG/MEG 


EEG is used in BCIs since it offers several advantages, such as being non- 
invasive, having high temporal resolution, and relatively low costs. In a BCI 
system, the EEG electrodes capture the brain's electrical activity, which 
amplifies those signals. Features such as event-related potentials (ERPs) and 
sensorimotor rhythms (SMRs) are extracted from the EEG signals to interpret 
the users' intentions into a command. For example, a P300-based BCI Speller 
used EEG to allow users to select letters on a screen by focusing on a character. 
These BCIs have been used to enable patients with motor speech impairments, 
such as those with amyotrophic lateral sclerosis (ALS), which will be 
discussed more in Chapter 12, to be able to communicate more efficiently 
[142], [143], [144]. The hardware used for EEG-based BCIs includes the 
electrode caps, amplifiers, and data acquisition units, as discussed in Chapter 
4. Fig. 6.2 [145] shows an EEG-based BCI system. 


JSMRI/NIRs 

fMRI can also be used in BCIs due to its high spatial resolution and ability to 
orient BCIs to specific brain areas associated with tasks or thoughts. Although 
fMRIs disadvantages of being expensive and size make it unpractical for BCIs 
for daily life activities, they can provide insights into the brain architectures 
and identify patterns associated with mental states and intentions. These 
insights, such as understanding brain areas related to imagining specific 
movements or objects, can help translate these signs into control commands 
for external devices [146]. The hardware used in {MRI-based BCIs includes 
the fMRI scanners, as discussed in Chapter 4. While EEG can offer real-time 
measurements with high temporal resolution, they lack the spatial precision of 
fMRI. On the other hand, fMRI can provide detailed spatial maps of brain 
activity but is limited by its temporal resolution and high costs. Leveraging 
both technologies can have the potential to provide a more comprehensive 
approach to BCI research and development since it can provide both precise 
temporal and spatial information about brain activity [147]. 

Capturing brain activity is only the first step in a successful BCI. Next, we will 
discuss what happens after this data capture and how these measurements of 
brain activity are processed to produce control commands to external devices. 
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Signal Processing and Feature Extraction 


Signal processing is crucial in translating raw brain signals into actionable 
commands controlling external devices. This process involves multiple steps, 
each critical for ensuring that the BCI system is reliable and accurate, allowing 
users to interact with the world in novel ways, especially in assistive 
technologies for individuals with disabilities. 

The first step in signal processing for BCIs is reducing noise from the raw 
signals. Brain signals, particularly those captured by EEGs or fMRI, are 
inherently noisy due to various factors, including physiological artifacts and 
environmental interferences. There are different ways to reduce noise. 
Independent Component Analysis (ICA) is a statistical technique that separates 
a multivariate signal into additive, independent components. It is particularly 
useful in isolating and removing artifacts, such as eye blinks, from EEG data 
without losing critical neural information. Common Average Referencing 
(CAR) is a spatial filtering technique where the average of the signals from all 
electrodes is subtracted from each electrode's signal. This method helps reduce 
common noise across the electrodes, enhancing the signal-to-noise ratio. 
Lastly, Adaptive filtering dynamically adjusts the filtering process based on 
the properties of the incoming signals. It effectively reduces noise that changes 
over time, such as slow drifts in EEG signals caused by sweat or electrode 
movement. These noise reduction techniques are essential because any 
residual noise can significantly impact the accuracy of the BCI, leading to 
incorrect interpretations of the user's intentions [148]. 

After noise reduction, features from the signal must be extracted to identify 
key brain signal characteristics corresponding to the users’ intentions. Time- 
domain, frequency domain, and time-frequency analyses are commonly used 
in feature extraction for BCIs. 


Time Domain Analysis 


Time-domain features such as amplitude, mean, and variance are extracted 
directly from the signal waveform. These features are often used in real-time 
BCIs because they are straightforward to compute and provide immediate 
insights into brain activity. 
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Frequency Domain Analysis 

Frequency-domain analysis involves transforming the time-domain signal into 
the frequency domain using techniques like the Fast Fourier Transform (FFT). 
This approach helps identify rhythmic activity, such as alpha, beta, and gamma 
waves, corresponding to different mental states or cognitive processes. 
Time-Frequency Analysis 

Time-frequency analysis combines the benefits of both time and frequency 
domain analyses. Techniques like the Short-Time Fourier Transform (STFT) 
or wavelet transform are used to analyze how the frequency content of a signal 
evolves. This is particularly useful for detecting transient events, such as motor 
imagery, where the brain's activity changes quickly. 

These features are then used to train machine learning algorithms (Chapter 3) 
to understand user intentions [149]. Supervises learning algorithms such as 
Support Vector Machines (SVM), Linear Discriminant Analysis (LDA), and 
Convolutional Neural Networks (CNN), which can be trained to label datasets 
to classify brain signals. 

Support Vector Machines (SVM): 

SVM is a popular machine learning algorithm that finds the optimal boundary 
(or hyperplane) separating different brain signal classes. It is particularly 
effective for binary classification tasks, such as distinguishing between "left" 
and "right" motor imagery. 

Linear Discriminant Analysis (LDA): 

LDA is a simple yet powerful classifier that assumes linear separability 
between classes. It works well when the brain signals exhibit a Gaussian 
distribution and is computationally efficient, making it suitable for real-time 
BCI applications. 


Convolutional Neural Networks (CNNs) 


CNNs, a type of deep learning algorithm, have shown significant promise in 
improving the accuracy of BCIs. CNNs are particularly effective at 
automatically learning complex patterns in high-dimensional data, such as 
EEG signals, without requiring extensive manual feature extraction. 
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Unsupervised learning methods can include clustering and Principal 
Component Analysis (PCA) to identify data patterns without predefined labels. 


Clustering 

Unsupervised clustering algorithms group brain signal data into clusters based 
on their inherent similarities without needing predefined labels. This approach 
can be useful for exploring unknown patterns in the data or identifying distinct 
mental states that were not initially considered. 


Principal Component Analysis (PCA) 

PCA is a dimensionality reduction technique that transforms the original 
features into a smaller set of uncorrelated components. This reduces the 
complexity of the data while retaining the most important information, making 
it easier for the BCI system to classify signals accurately. 

These algorithms often translate brain activity patterns into control commands 
[150]. Once brain activity is captured and interpreted, it can create commands 
for external devices. The machine learning models trained using these 
techniques are crucial for the BCI's ability to decode the user's intentions 
accurately. As BCI technology advances, deep learning models like CNNs are 
increasingly being used because of their ability to improve classification 
accuracy and handle the complex, non-linear nature of brain signals. 


Output Devices 


Various output devices can be used in BCIs, including computer cursors, 
robotic limbs, wheelchairs, and communication aids. These devices translate 
the brain signal into actions, enabling users to interact with their environment 
more independently. These devices include both hardware and software 
components. 


Hardware 

Hardware for output devices includes motors, sensors, and actuators that 
execute the appropriate commands [151]. 

Motors 


Motors are fundamental in BCI systems where physical movement is the de- 
sired output. These motors convert the electrical commands generated by the 
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BCI into mechanical movement. The choice of motor is crucial because it di- 
rectly affects the system's precision, speed, and responsiveness. High-preci- 
sion motors, such as servo and stepper motors, are commonly used in applica- 
tions where fine control is essential. For example, in a robotic limb controlled 
by a BCI, servo motors allow for precise control of joint angles, enabling the 
user to perform delicate tasks like picking up small objects or manipulating 
tools. In applications requiring significant force, such as controlling a BCI- 
powered exoskeleton, motors must provide adequate torque to support and 
move the user's body weight. Brushless DC motors are often used in these 
scenarios due to their high efficiency, power density, and smooth operation. 
The speed of the motor's response to BCI commands is another critical factor. 
Motors in BCI systems must quickly translate user intentions into movement 
to create a natural and intuitive interaction. This is particularly important in 
dynamic environments, such as BCI-controlled wheelchairs, where rapid ad- 
justments may be needed to avoid obstacles. 


Sensors 

Sensors play a vital role in providing feedback to the BCI system and ensuring 
accurate and safe operation of the output device. They monitor various aspects 
of the device's performance and the environment, allowing for real-time ad- 
justments. Sensors that measure joints’ position, angle, and velocity are crucial 
in robotic limbs and exoskeletons. These sensors provide feedback to the BCI 
system about the limb's current state, enabling precise control over movements 
and ensuring that the limb moves as intended by the user. These sensors meas- 
ure the force or pressure the output device applies. In a prosthetic hand, for 
instance, force sensors can ensure that the hand grips objects with appropriate 
strength, preventing damage to delicate items or injury to the user. 
Environmental sensors are essential for safe navigation for devices like BCI- 
controlled wheelchairs. Ultrasonic sensors, infrared sensors, and light detec- 
tion and ranging (LIDAR) can detect obstacles and measure distances, allow- 
ing the wheelchair to avoid collisions and navigate through complex environ- 
ments safely. These sensors work with the BCI to adjust the real-time move- 
ment path based on environmental conditions. In advanced prosthetics and ro- 
botic limbs, tactile sensors give the user feedback about objects' texture, shape, 
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and firmness. Haptic feedback mechanisms can simulate the sensation of 
touch, allowing users to "feel" what they are interacting with, even through a 
robotic device. This sensory feedback is critical for creating a more natural and 
effective BCI experience. 


Actuators 


Actuators are devices that convert electrical signals into mechanical move- 
ment. In BCI systems, actuators are responsible for executing the commands 
generated by the BCI and are central to the system's ability to perform tasks in 
the physical world. Linear actuators produce motion in a straight line instead 
of rotational motion. They are commonly used in applications like adjustable 
beds, robotic arms, and exoskeletons, requiring controlled, linear movement. 
Rotary actuators produce rotational motion and are essential in devices like 
BCI-controlled robotic arms and prosthetic limbs. These actuators enable 
joints to rotate smoothly and accurately, mimicking the natural movement of 
human limbs. Hydraulic or pneumatic actuators might be used in applications 
requiring powerful and precise movements, such as in heavy-duty exoskele- 
tons or robotic systems. These actuators provide high force and can support 
significant loads, making them suitable for tasks requiring strength and preci- 
sion. Microactuators are used in smaller, more delicate BCI applications, such 
as fine-tuned robotic hands or micro-manipulation devices. These tiny actua- 
tors allow precise control at small scales, enabling tasks requiring fine motor 
skills. 

The successful operation of a BCI system depends on the seamless integration 
of motors, sensors, and actuators. Each component must harmonize to translate 
the user's intentions into accurate, real-world actions. Advanced control sys- 
tems, often incorporating feedback loops, are required to manage the interac- 
tion between the BCI, motors, sensors, and actuators. Based on sensor feed- 
back, these systems adjust the output in real-time, ensuring smooth and accu- 
rate operation. In applications where safety is critical, such as BCI-controlled 
wheelchairs or exoskeletons, redundant sensors and actuators might be em- 
ployed to ensure reliable operation. If one component fails, the system can 
switch to a backup, preventing accidents or loss of control. Many BCI systems 
allow customization of the hardware components to better suit the individual 
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user's needs. For instance, the sensors' sensitivity, the actuators’ speed, and the 
motors' power can be adjusted to match the user's strength, dexterity, and spe- 
cific requirements. 

The hardware components of BCI output devices, including motors, sensors, 
and actuators, are essential for translating brain signals into meaningful phys- 
ical actions. These components must be carefully selected and integrated to 
ensure the BCI system is responsive, precise, and safe. As BCI technology ad- 
vances, hardware improvements will enable even more sophisticated and intu- 
itive control of external devices, expanding the possibilities for human-com- 
puter interaction and assistive technologies. 


Software 


Software used in BCIs is an essential component that bridges the gap between 
the user’s brain signals and the external devices they aim to control. The 
software must be highly responsive and adaptable to the unique needs of each 
user to facilitate a seamless and intuitive interaction with the BCI system. This 
adaptability is crucial for the software to accurately interpret and translate 
brain signals into actionable commands, ensuring the user can effectively 
operate the BClI-controlled devices (Fig. 6.3 [152]). BCI software is 
responsible for the real-time processing of brain signals, such as those captured 
by EEG or other neuroimaging techniques. The software applies algorithms to 
filter, clean, and analyze these signals to extract relevant features 
corresponding to the user’s intended commands. Once the brain signals are 
processed, the software uses machine learning and pattern recognition 
algorithms to classify the signals into specific commands. This step is critical 
for translating the user’s mental intentions into precise control actions for the 
output device. 

These software systems can include graphical user interfaces (GUIs) that can 
provide visual feedback to the user [153]. This feedback is crucial as it allows 
users to see the effects of their brain signals in real-time, helping them adjust 
their mental strategies to improve control. For instance, in a BCI used for 
communication, a GUI might display a virtual keyboard where the user can 
select letters or words using their brain activity. GUIs in BCI software are often 
customizable to accommodate different user needs and preferences. Users can 
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modify the display layout, color schemes, and feedback mechanisms to create 
a comfortable and effective interface. 

Additionally, BCI software usually contains features that allow users to 
calibrate and train the system. Calibration is a vital feature of BCI software, 
where the system learns to recognize the specific brain signal patterns of the 
user. During calibration, the user is guided through a series of tasks or 
exercises, and the software adjusts its algorithms to detect better and interpret 
the user's unique neural signatures. This step is crucial for improving the 
accuracy and reliability of the BCI system. Training is an ongoing process 
where users practice generating the specific brain signals required to control 
the BCI. The software often includes tools and exercises to help users improve 
their ability to produce clear and consistent brain signals. Over time, this 
training can enhance the user's proficiency in BCI, leading to more effective 
and seamless device control [142]. BCI software also often incorporates 
adaptive algorithms that continuously learn and refine their understanding of 
the user's brain signals. These algorithms can adjust in real-time based on the 
user's performance, making the system more responsive to changes in brain 
activity over time. This adaptability is especially important for users whose 
neural patterns may fluctuate due to fatigue, emotional state, or learning 
effects. The software can also make user-specific adjustments, such as 
adapting to the user’s pace of interaction or the difficulty level of tasks. This 
ensures that the BCI remains user-friendly and effective, even as the user’s 
needs evolve. 

Software platforms used in BCIs include the BCI2000 platform, which offers 
suites of tools for developing BCI supplication. BCI2000 is a widely used 
platform in the BCI community, offering a comprehensive set of tools for 
developing BCI applications. It supports various signal acquisition methods, 
including EEG, ECoG, and others, and provides signal processing, feature 
extraction, and classification modules. The versatile platform supports various 
BCI paradigms, including motor imagery, P300-based communication, and 
SSVEP (Steady-State Visually Evoked Potential) systems. This versatility 
makes BCI2000 a popular choice for both research and clinical applications. 
BCI2000 has a strong community of users and developers contributing to its 
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ongoing development, making it a robust and well-supported platform for BCI 
research. Another platform is OpenViBE, an open-source system that supports 
real-time processing for various BCIs [154], [155]. OpenViBE is an open- 
source platform designed to develop and test real-time BCI systems. Being 
open source, it allows developers to modify and extend the platform according 
to their specific needs, providing a high degree of flexibility. OpenViBE 
supports real-time signal processing, which is crucial for applications that 
require immediate feedback and interaction, such as neurofeedback or real- 
time control of assistive devices. The platform includes powerful data 
visualization and feedback tools, enabling users to monitor real-time brain 
signals and system performance. This is particularly useful in research settings 
and practical BCI applications, where understanding and quickly interpreting 
the data is essential. 


Types of BCIs 


Invasive vs Noninvasive 


Early in this chapter, it was mentioned that BCIs can be non-invasive or 
invasive (Fig. 6.4 [156]). This next section will discuss these different types of 
BCls, their hardware components, typical software, and their advantages and 


disadvantages. 
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Figure 6.3: Flow of information from brain activity to external devices for BCIs, from brain 
activity to commands [152] 
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Invasive BCIs 


Invasive BCIs require surgery to implant electrodes directly into brain tissue. 
This type of BCI is designed to capture high-resolution neural signals from 
specific brain regions, resulting in detailed and accurate data. The hardware 
components for invasive BCIs include microelectrode arrays, implantable 
sensors, and wireless transmission units. The materials used for these invasive 
BCIs must be biocompatible to mimic any adverse effects after implantation 
[157]. BCI systems can use electrocorticography (ECoG) grids placed on the 
brain's surface to capture electrical activity for motor commands for prosthetic 
devices or stereo electroencephalography (SEEG) that involves inserting 
electrodes into specific regions of deep brain structures [155]. Other systems 
can use intracortical implants, such as the invasive BCI in the BrainGate 
system. This BCI system uses an array of microelectrodes implanted in the 
motor cortex to control robotic limbs or computer cursors for those with 
paralysis [137]. Invasive BCIs offer a high spatial and temporal resolution. 
However, they come with significant risks. These risks can include infection, 
inflammation, and long-term stability issues. Additionally, surgical 
implantation of a BCI can pose ethical and medical challenges [158]. 


Noninvasive BCIs 


Noninvasive BCIs do not require surgical procedures and require external 
devices such as 
EEG, which is the Non invasive 


most used, or MEG. EEG 
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use the hardware meas 
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Figure 6.4: Non-invasive vs. Invasive BCI systems [156] 


and gaming [142]. 
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With no need for surgery, noninvasive BCIs are much more accessible and 
safer. However, they have lower signal quality and resolution and have more 
challenges with signal processing due to more noise [159]. 


Ethical Considerations and Future 
Implications 


Now that we have explored the fundamentals of BCIs, it is important to 
consider ethical considerations surrounding this technology and how this type 
of technology can impact society. 


Privacy and Security 


Ethical concerns of BCI systems include data privacy issues. Since BCIs 
capture susceptible neural activity data, serious privacy concerns exist. This 
data type may reveal intimate details about a person, such as their thoughts, 
emotions, and intentions. Additionally, issues can surround who owns this data 
and how to use or share it. It is essential that BCI system data is protected from 
unauthorized access and that there are strict privacy policies and regulations 
to safeguard users’ information [160]. Another concern surrounding the BCI 
system is security and preventing hacking by unauthorized personnel. Since 
BCIs have the potential to influence physical actions directly, a security breach 
of the system can have severe consequences. 

Additionally, the potential ability to manipulate thoughts or emotions through 
BCIs can challenge the notion of free will and personal identity. Other security 
concerns include mind control and surveillance. There must be extensive 
security measures such as encryption, secure data protocols, and robust 
authentication methods to protect the system from cyber threats [161],[162]. 


Accessibility and Medical Ethics 


BCI technology has enormous potential to enhance the quality of life for 
individuals with disabilities. However, the accessibility of these technologies 
is limited due to their high cost and complexity. Additionally, hardware and 
software systems used in BCIs contribute to their accessibility. To bridge the 
gap between cutting-edge technology and its use in daily life, BCIs require 
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hardware and software systems that are easy to use and focused on user- 
centered design. Developing affordable hardware, user-friendly interfaces, and 
training programs are all additional efforts that are being made to reduce the 
cost of and simplify BCI systems to make them more accessible. Even when 
accessible, medical use of BCIs raises ethical concerns such as informed 
consent. Additionally, the potential for unintended consequences due to the 
risk of surgical implantation also raises concerns. Ethical review and oversight 
are required to ensure that these technologies are used responsibly, and that 
patient well-being is prioritized [163], [164]. 


Potential Societal Impact 


The widespread adoption of BCIs can have immense effects on society. Since 
BCIs have the potential to enhance human capabilities, their use can lead to 
debates about what it means to be human. Additionally, due to accessibility 
concems, BCIs can lead to new forms of inequality where those with access 
may gain significant advantages over those without access [160]. On the other 
hand, BCI technology can open new avenues of communication, learning, and 
interaction. Additionally, they shift cultural and social norms, leading to new 
forms of social interaction and collaboration. Proactive planning and 
inclusivity can help ensure we can maximize the benefits of BCI technology 
and minimize potential negative consequences [165]. 


Future Directions and Challenges 


As we conclude this chapter, it is important to assess the future directions and 
challenges associated with the advancement of BCI technology. This field is 
rapidly evolving with several trends that can change how we interact with 
technology. One significant modern trend is the integration of BCIs with 
virtual or augmented reality, which can enhance immersive experiences for 
gaming and rehabilitation [166]. Additionally, noninvasive BCIs are being 
used outside clinical settings, which makes them more accessible to more 
users. Another modern trend is their use in mental health applications, such as 
detecting and modulating emotional states. This could pave new ways for 
treating mental health conditions such as depression and anxiety [150]. 
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Miniaturization of hardware, such as microelectrode arrays and flexible 
electronics and software innovations, is also needed for the future development 
of BCIs to make them more seamless and accessible. Despite these 
advancements, there are challenges in scaling the widespread adoption of 
BCIs. This challenge can arise due to the high cost of BCIs. Additionally, there 
are challenges in ensuring they are reliable in diverse settings where signal 
quality can be affected [143], [163]. 


Chapter 6: Summary 


In this chapter, we explored the world of BCIs, discussing their components, 
technologies, and ethical considerations. We assess how brain signals are 
captured and decoded and then used to produce commands. Through this 
journey of understanding this technology, you have gained a comprehensive 
view of how BCIs are used and how they can influence the future of human- 
computer interaction. As you look forward to your neural engineering journey, 
considering the potential and ethical impacts of BCIs is essential to ensure that 
we are equitable and responsible in the technologies you might create so that 
their full potential can be reached to benefit society. 

In the next chapter, you will transition into the exciting field of 
neurostimulation, which explores how electrical stimulation of the brain can 
treat various neurological conditions and enhance cognitive functioning. You 
will explore two neurostimulation techniques: Deep Brain Stimulation (DBS) 
and Transcranial Magnetic Stimulation (TMS). These technologies can offer 
complementary approaches to BCIs, further expanding your knowledge of 
understanding and interaction with the brain. 
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Chapter 6: Learning Activities 


Learning Activity 6.1 
Basic Architecture of a BCI System 
Objective 


Understand and design the basic architecture of a Brain- 


Computer Interface (BCI) system by creating a functional block diagram 


(FBD), comparing it with a reference diagram, and identifying differences, 


similarities, and potential improvements. 


Materials Needed 


Computers or tablets with presentation software (e.g., PowerPoint) 
Internet access to view the reference diagram 

Access to the article: Brain-Computer Interface (BCI) System Func- 
tional Block Diagram 


Step-by-Step Instructions 


Il. 


Form Groups: 
e Divide the class into pairs (groups of 2) to encourage collabora- 
tion. 
Introduction: 
e Briefly introduce the concept of Brain-Computer Interfaces 
(BCIs) and their applications. 
e Explain the importance of understanding the architecture of BCI 
systems. 
Activity Part 1 - Create a Functional Block Diagram: 
e Each group will create a PowerPoint presentation with a func- 
tional block diagram (FBD) of a BCI system. 
e The FBD should include the following basic components: 
o Signal Acquisition 
o Signal Processing (including feature extraction and clas- 
sification) 
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o Translation Algorithm 
o Control Interface (to external devices) 

e Encourage students to think about how data flows through the 
system, from brain signal acquisition to the control of external 
devices. 

4. Activity Part 2 - Present and Share: 


e After creating their FBDs, each group will present their 
diagrams to the class. 
e Discuss the different approaches taken by each group. 
e Activity Part 3 - Compare with Reference Diagram: 
e Display Figure 2 from the article: General Functional Block 
Diagram of a Brain-Computer Interface. 
e Give students time to study the reference diagram. 
e Activity Part 4 - Analyze and Reflect: 
e Students will compare their FBDs with the reference 
diagram in their groups. 
e Answer the following questions: 

* What is missing? Identify any key components or 
connections in the reference diagram that are absent 
in their diagrams. 

" How is it different? Note any structural or 
conceptual differences between their FBDs and the 
reference diagram. 

" How is it similar? Highlight the similarities in 
components and data flow between their FBDs and 
the reference diagram. 

= How would you change it? Based on the 
comparison, discuss potential improvements to their 
diagrams. 

e Group Discussion: 
e Come together as a class and discuss the findings from each 
group. 
e Encourage students to share insights and reflections on their 
observed differences and similarities. 
e Discuss the importance of each component and the overall 
system integration in BCI design. 


277 


e Conclusion: 

e Summarize the key takeaways from the activity. 

e Emphasize the importance of a well-designed architecture in 
the functionality and efficiency of BCI systems. 

e Follow-Up Assignment: 

e Each student writes a short reflection (1-2 pages) on what 
they learned about BCI systems from this activity, including 
any challenges faced during the diagram creation and 
comparison process. 


This activity will enhance students’ understanding of the basic architecture of 
BCI systems and develop their skills in collaborative work, critical thinking, 
and technical communication. 


Learning Activity 6.2 


Ethical Repercussions of Brain-Computer 
Interfaces (BCIs) 


Objective 
Students will explore and understand the ethical implications of Brain- 
Computer Interfaces (BCIs) through individual brainstorming, collaborative 
discussion, and analysis of real-world applications. 


Materials Needed 


e Pen and paper or digital devices for note-taking 

e Whiteboard or digital collaboration tool (e.g., Google Docs, 
Jamboard) 

e Access to the article "The ethics of gaming with brain-computer 


interfaces" 
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Activity Outline 


Part 1: Individual Brainstorming (15 minutes) 


1. Introduction (5 minutes): 


Briefly introduce the concept of BCIs and their current and 
potential applications in various fields such as medicine, gaming, 
and communication. 

Explain the goal of the activity: to identify and discuss the ethical 
repercussions of using BCIs. 


2. Brainstorming Session (10 minutes): 


Instruct students to spend 10 minutes listing as many neuro-ethical 
repercussions of BCIs. Encourage them to consider privacy, 
consent, security, accessibility, and the potential impact on 
society. 


Part 2: Collaborative Discussion (20 minutes) 


1. Sharing and Combining Lists (15 minutes): 


Have students pair up or form small groups (3-4 students) and 
share their lists. 

Each group should combine their lists into a comprehensive list of 
ethical issues, eliminating duplicates and refining the wording. 


2. Class-Wide Discussion (5 minutes): 


Bring the class together and have each group share their combined 
list. 

Compile a master list on the whiteboard or a shared digital 
document, ensuring all unique points are included. 


Part 3: Article Reading and Reflection (25 minutes) 
1. Reading the Article (10 minutes): 


Direct students to read the article "The ethics of gaming with 


brain-computer interfaces." They can read individually or in pairs. 


2. Reflection and Discussion (15 minutes): 


After reading, ask students to reflect on the article and identify 
what surprised them the most. 
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e Facilitate a class discussion where students share their thoughts 
and surprising elements from the article. 

e Encourage students to compare the ethical issues mentioned in the 
article with those on the master list created earlier. 


Part 4: Ranking Ethical Issues (20 minutes) 


1. Individual Ranking (5 minutes): 
e Ask students to individually rank the ethical issues listed on the 
master list from most to least critical based on their perspective. 
2. Group Consensus (10 minutes): 
e Form small groups again and have them discuss their rankings. 
e Each group should reach a consensus on a ranked list of ethical 
issues. 
3. Class Discussion and Final Ranking (5 minutes): 
e Have each group share their ranked list with the class. 
e Facilitate a discussion to create a final class-ranked list of ethical 
issues, considering the arguments presented by each group. 


Conclusion (5 minutes): 


e Summarize the key points discussed during the activity. 

e Highlight the importance of considering ethical implications in neu- 
roengineering and encourage students to continue thinking critically 
about these issues as they progress in their studies. 


Assessment: 


e Participation in brainstorming, group discussions, and class 
discussions. 
e Quality and depth of the individual and group lists of ethical issues. 
e Reflecting thoughtfully on the article and ranking ethical issues, 
This activity promotes individual critical thinking, collaborative learning, and 
the ability to articulate and debate ethical considerations in a scientific context. 
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Learning Activity 6.3 
Quescussion on "BCI is Better than BMI" 


Objective 


Students will explore the relative merits and limitations of 
Brain-Computer Interfaces (BCI) and Brain-Machine Interfaces (BMI) 
through a structured question-based discussion, enhancing their understanding 
of neuroengineering concepts. 


Duration: 


20 minutes 


Materials Needed 


e Whiteboard and markers 
e Sticky notes 
e Timer 


Activity Outline: 


1. Introduction (3 minutes): 

e Briefly introduce BCIs (Brain-Computer Interfaces) and BMIs 
(Brain-Machine Interfaces), highlighting their definitions, appli- 
cations, and key differences. 

e Explain that BCIs provide a direct communication pathway be- 
tween the brain and an external device, while BMIs can imply in- 
terfaces with machines or robotic systems. 

2. Setup for Quescussion (2 minutes): 

e Explain the rules of the Quescussion: 

e Only questions are allowed. Statements or answers are not 
permitted. 

e Questions should provoke thought, challenge assump- 
tions, and explore different angles of the topic. 

e Students take turns asking questions, and no question can 
be repeated. 
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e Ifa student accidentally makes a statement, they will be 
asked to rephrase it as a question or pass their turn. 
e Provide each student with sticky notes and a pen. 
3. Quescussion (10 minutes): 
e Start the timer for 10 minutes. 
e Students take turns asking questions about why BCIs might be 
considered better than BMIs. 
e Encourage students to write down thought-provoking questions 
on sticky notes and place them on the whiteboard. 
e The instructor moderates to ensure adherence to the rules and to 
keep the discussion focused and dynamic. 


1. Reflection and Discussion (3 minutes): 

o Once the Quescussion is complete, review the questions 
posted on the whiteboard as a class. 

o Facilitate a brief discussion where students can make 
statements and reflect on the questions raised. Encourage 
them to discuss: 

" Key insights gained from the Quescussion. 
«Which questions were the most challenging or eye- 
opening? 
" How the activity influenced their understanding of 
BCIs and BMIs. 
2. Conclusion (2 minutes): 

o Summarize the main points discussed. 

o Highlight any consensus reached or major differences that 
remained unresolved. 


@ 


eee 
a2 
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Chapter 6: Lab introduction 


In this series of lab exercises, you will explore the principles and 
practical applications of Brain-Computer Interfaces (BCIs) through 
hands-on activities using MATLAB, Simulink, and Python. These 
labs will provide you with experience in running and simulating BCI 
systems, focusing on key processes such as neural activity detection, 
feature extraction, and classification. 

In the first lab, you will utilize MATLAB and Simulink to operate a 
basic BCI system. This exercise will guide you through essential 
steps in a typical BCI workflow, including detecting neural activity, 
amplifying signals, reducing noise, extracting features, and 
generating output commands. This practical experience will help 
you understand how BCIs process neural signals to achieve 
functional outcomes. 

In the second lab, you will simulate EEG signals and implement a 
simple BCI system using Python. The lab will enable you to 
interactively adjust parameters such as noise levels and stimulation 


intensity, observing their effects on EEG signals in real-time through 


a graphical user interface (GUI). This hands-on simulation will 
enhance your understanding of how feature extraction and 
classification are applied in BCI systems, providing a 
comprehensive overview of their functionality and practical use. 
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Chapter 6: Lab Example 1 


We will use MATLAB and Simulink to run a simple BCI 
system in this lab example. We will go through the 
processes of a typical BCI system, such as neural activity 


detection, amplification, noise reduction, feature extraction, 
and an output command. 

Steps 

First, follow this link. 


EEG Feature Extraction Toolbox - File Exchange - MATLAB Central 
(mathworks.com) 


Download the EEG Feature Extraction Toolbox. Make sure that this toolbox is 
unzipped and in your MATLAB drive. Then, navigate to the GitHub page for 
this textbook and go to Chapter 6, Lab Example 1. You will find two MATLAB 
files: one called depressed_signals.m and the other healthy_signals.m. You 
want to download both of those files. Open them in MATLAB and run each of 
them. This MATLAB script generated artificial EEG signals that produced 
EEG signals from healthy controls, and then EEG signals that exhibited 
depression-related features. After this, you should see two variables in your 
Workspace: heatlhy signals and depressed signals. These variables in your 
workspace are essential for the Simulink model to work correctly. Now, 
navigate to the Home tab and open up Simulink. 

Once in Simulink, navigate to the Library Browser and insert the following 
boxes: two signals from workspace boxes, two Bandpass filter boxes, a 
MATLAB code box, two scope boxes, and two lamps. 
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For the two Signal From 
double- 
click them, and under Signal, 


Workspace boxes, 


type healthy signal for one 
box and depressed signal for 
the other. These boxes will 
take the EEG data generated 
by the first MATLAB code 
given at the beginning of the 
example. You then want to 
connect one of these boxes to 
the Bandpass Filter boxes. 
These filter boxes will clean 
up the EEG signal and reduce 
the amount of noise. Double- 
click your Bandpass Filter 
Box and set each to the 
parameters in Fig. 6.5. 


[Pa] Block Parameters: Bandpass Filter 
Bandpass Filter 
Design a bandpass filter. 


Filter specifications 
Impulse response: FIR 
Order mode: 
Filter type: 


Minimum 
Single-rate 
Frequency specifications 
Frequency units: Hz Input sample rate: 
Stopband frequency 1: 7 
Passband frequency 2: 30 
Magnitude specifications 
Magnitude units: dB 
Stopband attenuation 1: 60 


Stopband attenuation 2: 60 


Algorithm 

Design method: Equiripple 
> Design options 

Filter implementation 
Structure: Direct-form FIR 

C Use basic elements to enable filter customization 
Input processing: Columns as channels (frame based) 


OC Use variable names for coefficients 


Cox) 


Cancel 


Figure 6.5: Bandpass filter parameters 


Passband frequency 1: 
Stopband frequency 2: 


__ Passband ripple: 1 


View Filter Response | 


1000 
8 
31 


Help Apply 


Then, connect both Bandpass Filter boxes to the MATLAB code box. The 
MATLAB code box is then attached to each of the scope boxes. In your 
MATLAB code box, insert the following code. 


function [healthyPulseControl, depressedPulseControl] = 
processEEGAndDeterminePulse(healthy_eeg data, 


depressed _eeg data) 


% Define opts structure with all necessary fields 


before using it 
opts = 


struct('fs,' 500, 


‘alpha,’ 2); 


% Process healthy EEG data 
healthyPulseControl = 
processSingleEEG(healthy_eeg data, opts); 


% Process depressed EEG data 
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depressedPulseControl = 
processSingleEEG(depressed_eeg data, opts); 
end 


function pulseControl = processSingleEEG(eeg data, opts) 

% Check if jfeeg function is compatible with MATLAB 
Coder. 

% If it's not, you would need to replace it with 
equivalent functionality 

% that is compatible with MATLAB Coder. 


% Hjorth Activity, Mobility, Complexity 
f1 = jfeeg('ha', eeg data, opts); 
f2 = jfeeg('hm', eeg data, opts); 
f3 = jfeeg('hc', eeg data, opts); 


% Feature vector for Hjorth features (not used 
further in this function) 
7 OPE wee = |r, TAs wells 


% Band Power Alpha 
f4 = jfeeg('bpa', eeg data, opts); 


% Tsallis Entropy 
f5 = jfeeg('te', eeg data, opts); 


% Define threshold values for determining 
pulseControl 

threshold_activity = 0.1; % Adjust as needed 

threshold_tsallis entropy = 0.2; % Adjust as needed 


% Evaluate the condition for pulse generation 
pulseControl = (f1 > threshold_activity) && (f5 > 
threshold_tsallis_ entropy); 
end 
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Figure 6.6: Final Simulink Model 


This code is the signal processing and feature extraction part of our BCI. This 
is what your BCI should look like in Simulink. For the lamps, double-click 
them. Under the Main tab, make sure to have a State of 0, which is the Color 
red, and a State of 1, which is the Color green. You want to ensure that one 
lamp is connected to the scope box with the healthyPulseControl variable and 
the other to the depressedPulsecontrol. To connect the lamp, click the chain 
icon above the lamp and click on the appropriate scope box. Your final BCI 
Simulink model should look like Fig. 6.6. 

After you have created your Simulink BCI model, select run. Depending on 
your computer, this simulation may take a few minutes to run, so don’t be 
alarmed if it takes a few minutes. What we expect to see is that our BCI can 
distinguish between brain activity that is either healthy or exhibits features of 
depressed neural activity. Once the BCI detects that brain acidity is exhibiting 


3 @ 


healthy_signals Bandpass 
filterededg_data 
| 


Signal From Bandpass Filter l healthy_eeg data healthyPulseContro! 
Workspace 


processEEGAndDeterminePulse 


depressed_signals Bandpass =a 
depressed_eeg_data depressedPulseControl 


Signal From 
Workspace1 


Bandpass Filter! 


Figure 6.7: Model Results 
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features of depression, the BCI will generate a neurostimulation impulse to the 
neurons to try and correct this activity. Our simulation is successful if our 
healthyPusleControl scope box has no pulse generation and our 
depressedPulse control does. Additionally, the lamp connected to the 
depressedPulseControl should light up green when the pulse is delivered, and 
the other lamp should stay red (fig. 6.7). If this simulation happens too quickly, 
you can use the Step Forward button to see the color change. 


Other Examples and Resources 


Several other examples are available for you to use to keep exploring BCI 
systems. One includes a hardware example using Muse 2016. 


https://github.com/NeuroTechX/bci- 
workshop/blob/master/INSTRUCTIONS.md 


For more examples of practical applications of this chapter's content, please 
visit the book's dedicated GitHub repository page using this link. 


https://github.com/bddupre92/Neurobook_ BME UND 


A 
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Chapter 6: Lab Example 2 
EEG Signal Simulation and Classification 


Overview 


This lab example demonstrates the simulation of EEG 
signals, feature extraction, and classification using a simple Brain-Computer 
Interface (BCI) system implemented in Python. The simulation allows users to 
interactively modify parameters such as noise level and stimulation intensity 
and observe their effects on EEG signals in real-time through a graphical user 
interface (GUI). 


Goals of the Lab 


e Understand how EEG signals can be simulated and modified to reflect 


different conditions (healthy vs. depressed). 

e Learn how to extract features from EEG signals and classify them 
using machine learning techniques. 

e Gain hands-on experience with a simple BCI system through an 
interactive GUI. 

e Interpret the EEG signal processing results and understand the impact 
of the different parameters on signal classification. 


Setup 


Requirements 


e Python 3.8 or later 
e The following Python libraries: 


o numpy 
o matplotlib 
o scipy 


o  scikit-learn 
e  Tkinter (usually included with standard Python installations) 


Installation 


1. Clone the repository. 
2. Navigate to the project directory. 
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3. Install the required packages: 
e pip install -r requirements.txt 
Installing Tkinter 
Tkinter is usually included with standard Python installations. If you encounter 
issues, follow the instructions below: 
Windows 
Tkinter is included with the Python installer on Windows. If not available, 
reinstall Python using the official installer from python.org. 
macOS 
Tkinter is included with the Python installation on macOS. If not available, 
reinstall Python using the official installer from python.org. 
Linux 
On Debian-based distributions (e.g., Ubuntu): 
sudo apt-get install python3-tk 


On Red Hat-based distributions (e.g., Fedora): 
sudo dnf install python3-tkinter 


Using the GUI 
Running the GUI 


To run the main script and launch the GUI, execute: 
python main.py 


Interacting with the GUI 
1. Noise Level: Adjust the level of noise added to the simulated EEG 
signals. Higher noise levels simulate more real-world conditions with 
various interferences. 
2. Stimulation Intensity: Adjust the intensity of the electrical 
stimulation applied to the EEG signals. Higher intensities can show 
different patterns in the EEG signals. 
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After adjusting the parameters, click on Run Simulation to generate EEG 
signals, classify them, and observe the results. 


Meaning of the Parameters 


1. Noise Level: 
o Higher noise levels introduce more random variations to the 
EEG signals, simulating real-world conditions. 
o Lower noise levels produce cleaner signals, making it easier 
to distinguish between healthy and depressed conditions. 
2. Stimulation Intensity: 
e Higher stimulation intensities can significantly alter the EEG 
signal patterns, making the changes more pronounced. 
e Lower stimulation intensities might show subtler changes in the 


EEG signals. 
G Advanced BCI Simu... O x 
Noise Level: 0.1 


Stimulation Intensity: 5.0 


Run Simulation 


Figure 6.8: Meaning of Parameters 


How This Python Project Relates to BCI 

This project simulates a simple BCI system where EEG signals are generated, 
processed, and classified to determine if they reflect healthy or depressed 
neural activity. The classification results can then be used to create output 
commands, such as stimulating neurons to correct detected anomalies in brain 
activity. This hands-on experience helps me understand the basic components 
and functioning of BCIs. 


File Structure 


3. simulate _eeg.py: Contains functions to generate and plot simulated 
EEG signals. 


291 


4. extract_features.py: Contains functions to extract features from EEG 
signals and train a classifier. 

5. generate _output.py: Contains functions to generate output 

commands based on classifier predictions. 

bci_gui.py: Contains the GUI for interactive parameter adjustment. 

main.py: Launches the GUI for interactive use. 

requirements.txt: Lists all necessary packages to run the scripts. 

README.md: Provides instructions for setting up and running the 

project. 


own 


Interpretation of Results 

e EKG Signal Plots: The GUI plots the EEG signals for healthy and 
depressed conditions. Differences in amplitude and patterns can 
indicate variations in neural activity. 

e Classifier Accuracy: The accuracy of the classifier in distinguishing 
between healthy and depressed EEG signals is displayed. Higher 
accuracy indicates a better-performing classifier. 

e Output Commands: The generated output commands based on the 
classifier's predictions are shown. "Stimulate" indicates the detection 


Healthy EEG Signals 


Amplitude 
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Depressed EEG Signals 


Amplitude 


i) 200 400 600 800 1000 
Time (samples) 


Figure 6.9: Healthy vs Depressed EEG Signals 
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of depressed neural activity requiring intervention, while "No 
Stimulate" indicates healthy activity. 


Suggestions for Parameters to Modify 
Noise Level: 

e Increasing the noise level will make it harder for the classifier to 
distinguish between healthy and depressed signals, simulating more 
challenging real-world conditions. 

e Decreasing the noise level will produce cleaner signals, making it 
easier for the classifier to perform accurately. 

e Stimulation Intensity: 

o Higher intensities can lead to more pronounced changes in the 
EEG signals, helping to understand the effect of different 
stimulation strengths. 

o Lower intensities show subtler effects, which is useful for 
studying the minimal effective stimulation required for 
therapeutic effects. 

By exploring these parameters and their impacts on the EEG signals and 
classifier performance, users can gain deeper insights into the dynamics of 
neural activity and the potential of BCIs in clinical and research settings. 


A 
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Chapter 7 


Electrifying Insights in 
Neurostimulation 
294 


Introduction and Learning Objectives 


Neurostimulation technologies have become essential tools in treating various 
neurological disorders. By directly interacting with the nervous system, these 
technologies offer new ways to manage conditions that were previously 
difficult to treat with conventional therapies. This chapter will explore two 
prominent neurostimulation techniques: Transcranial Magnetic Stimulation 
(TMS) and Deep Brain Stimulation (DBS). By the end of this chapter, you will 
be able to: 


Understand the Importance of Neurostimulation Technologies: 
Recognize the crucial role of neurostimulation technologies in 
managing conditions resistant to conventional therapies. 

Describe the Components and Function of TMS and DBS Systems: 
Identify and explain the critical hardware components and setup 
processes for TMS and DBS systems. 

Utilize TMS and DBS Software Tools: Gain knowledge about the 
software tools for programming stimulation patterns, ensuring 
safety, and tracking patient responses in both TMS and DBS. 
Evaluate the Advantages and Disadvantages of TMS and DBS: 
Assess the pros and cons of TMS and DBS, considering factors such 
as non-invasiveness, effectiveness, and potential risks. 

Discuss the Current State of the Art and Future Directions in 
Neurostimulation: Stay informed about the latest advancements, 
ethical considerations, and potential new applications in 
neurostimulation technologies. 


Importance of Neurostimulation Technolo- 
gies in Treating Neurological Disorders 


Neurostimulation technologies are revolutionizing the field of neurology by 
providing treatments for a diverse range of disorders, including depression, 
epilepsy, Parkinson's disease, and chronic pain. These innovative technologies 
modulate neural activity, offering the unique advantage of restoring normal 
brain function or alleviating symptoms. Their significance is underscored by 
their ability to provide minimally invasive, targeted treatments with 
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significantly fewer side effects than traditional drug therapies. This makes 
neurostimulation technology particularly valuable for patients who do not 
respond well to conventional therapies or experience severe side effects, 
ultimately improving patient outcomes [167]. 


Overview of TMS and DBS 


Transcranial Magnetic Stimulation (TMS) and Deep Brain Stimulation (DBS) 
are two widely used neurostimulation techniques with unique mechanisms and 
applications. TMS is a non-invasive procedure that employs magnetic fields to 
stimulate nerve cells in the brain. It is primarily used for treating major 
depressive disorder, with ongoing research into its effectiveness for anxiety, 
stroke rehabilitation, and migraines. During TMS, an electromagnetic coil 
against the scalp generates magnetic pulses that induce electrical currents, 
modulating neuronal activity in specific brain regions [168]. On the other 
hand, DBS involves surgically implanting electrodes into precise areas of the 
brain. These electrodes are connected to a pulse generator in the chest that 
delivers electrical impulses. This method is commonly used to treat movement 
disorders such as Parkinson's disease, essential tremor, and dystonia. It has 
shown promise in managing psychiatric conditions like obsessive-compulsive 
disorder and depression [167]. Although DBS is more invasive than TMS, it 
offers the advantage of continuous, adjustable stimulation. Both TMS and 
DBS are significant advancements in neuroengineering, providing powerful 
tools to modulate brain activity and improve patient outcomes, marking a 
substantial impact on modern neurotherapeutics [169]. 


Transcranial Magnetic Stimulation (TMS) 


Transcranial Magnetic Stimulation (TMS) is a non-invasive technique that 
uses magnetic fields to stimulate nerve cells in the brain. This section delves 
into the hardware and software components of TMS systems, their setup and 
configuration, and the advantages and disadvantages of TMS. 
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TMS Mechanism 


TMS generates magnetic fields that penetrate the scalp and skull to induce 
electrical currents in specific brain areas. The TMS coil, typically in a figure- 
eight shape, is positioned over the target brain region. When activated, the coil 
generates a magnetic field that induces electric currents in the cortical neurons, 
leading to neuronal activation or inhibition. 

While TMS can stimulate motor pathways, as seen when a corticospinal volley 
activates spinal motoneurons to produce an action potential in the peripheral 
nerve, it can also target other types of nerve cells involved in various brain 


Simplified scheme of mechanism of action of TMS of the motor cortex 


Cortical interneuron TMS figure-of-eight coil 


Corticospinal tract Corticospinal volley induced by TMS 
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Peripheral nerve 
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is used to estimate corticospinal 
EMG activity tract excitability 


Figure 7.1: Simplified scheme of the mechanism of action of TMS on the motor cortex. The TMS 
coil generates a corticospinal volley that travels through the corticospinal tract, activating spinal 
motor neurons and induces an action potential in the peripheral nerves, leading to muscle activa- 
tion. This mechanism underlies the therapeutic effects of TMS. [171] 
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functions, depending on the stimulation site and parameters. The action 
potential ultimately leads to muscle activation in the contralateral target 
muscle [170]. Fig. 7.1 [171] provides a simplified overview of the TMS 
mechanism of action, focusing on the motor cortex. This process underlies the 
therapeutic effects of TMS in various neurological and psychiatric conditions, 
providing a non-invasive method to modulate brain activity and improve 
patient outcomes. 


Bingnetic field Electric field 


Left dorsolateral 
prefrontal cortex 


TMS target point 


Motor cortex 


Figure 7.2: Components of TMS [172] 


TMS Hardware 


TMS systems have several key components that deliver precise magnetic 
stimulation to targeted brain regions. The main components include the pre- 
planning and control unit, which is responsible for planning the stimulation 
parameters and controlling the overall operation of the TMS system. This unit 
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interfaces with the signal generator and the positioning system to ensure 
accurate delivery of the magnetic pulses [168]. 

TMS systems have several key components that deliver precise magnetic 
stimulation to targeted brain regions. The main components include the pre- 
planning and control unit, signal generator, positioning system, magnetic coil, 
and stereo vision camera. The pre-planning and control unit is responsible for 
planning the stimulation parameters and controlling the overall operation of 
the TMS system. This unit interfaces with the signal generator and the 
positioning system to ensure accurate delivery of the magnetic pulses. The 
signal generator generates the electrical signals that drive the magnetic coil. 
This component can produce pulses of varying intensity and frequency, which 
is crucial for therapeutic applications. Accurate positioning of the magnetic 
coil is essential for effective stimulation, and the positioning system, often 
guided by a stereo vision camera, ensures that the coil is placed at the correct 
location and orientation relative to the patient's head. The magnetic coil is the 
core component of the TMS system. It generates magnetic fields that penetrate 
the scalp and skull to reach the underlying brain tissue. The figure-eight coil 
design is commonly used for its ability to focus the magnetic field on a specific 
brain region. Additionally, the stereo vision camera assists in precise coil 
positioning by providing real-time visual feedback to the positioning system, 
helping to maintain the coil's correct placement during the TMS session. Fig. 
7.2 [172] illustrates the interaction between these components. 


Setup and Configuration 


The setup and configuration of a TMS system involve several steps to ensure 
accurate and effective stimulation (Fig. 7.3 [170]). The clinician inputs the 
desired stimulation parameters into the control unit, including the intensity, 
frequency, and duration of the magnetic pulses. Using the stereo vision camera 
and positioning system, the clinician accurately places the magnetic coil over 
the targeted brain region, ensuring the coil maintains the correct orientation 
and distance from the scalp [168]. 

The signal generator then produces the electrical signals that drive the coil, 
generating magnetic pulses that stimulate the underlying brain tissue and 
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modulate neuronal activity in the targeted area. Throughout the session, the 
clinician monitors the patient's response and makes any necessary adjustments 
to the stimulation parameters to optimize the therapeutic effect. 


Pre-planning Signal 
and control generator 


System for 


positioning 


Stereo vision 
camera 


= Electrical 
—<—-= Physical 


Figure 7.3: Block diagram of TMS hardware setup showing the interaction between the pre- 
planning and control unit, signal generator, positioning system, magnetic coil, and stereo vision 


camera [170] 


TMS Software 

TMS software is crucial in ensuring the precise and safe administration of 
magnetic stimulation. The software includes signal processing tools, analysis 
software, and patient response tracking mechanisms. The signal processing 
tools are essential for programming the stimulation patterns, ensuring safety, 
and conducting compliance checks. These tools allow clinicians to set the 
intensity, frequency, and duration of the magnetic pulses. Safety features are 
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Figure 7.4: TMS-targeted region neuronavigation and coil placement. The specific location of 
the TMS coil relative to the targeted brain region is shown using MRI guidance. a) 3D model 
of the head with the TMS coil positioned. b) Coil placement relative to the brain’s surface. c) 
MRI-guided targeting of the brain region for TMS. d) Coronal MRI slice showing the TMS 
target area. e) Axial MRI view showing coil alignment with the brain target. f) Sagittal MRI 
slice demonstrating coil position relative to the brain. [169] 


integrated to monitor the parameters continuously, ensuring they remain within 
safe limits to prevent adverse effects. 

TMS systems often integrate analysis software with imaging software to 
provide a comprehensive view of brain activity. This integration allows for the 
creation of detailed maps of brain regions, facilitating targeted stimulation 
[173]. The analysis software processes data from imaging techniques such as 
MRI or CT scans to identify the precise locations for stimulation, enhancing 
the accuracy and effectiveness of the treatment. The following image (Fig. 7.4 
[169]) illustrates TMS-targeted region neuronavigation and coil placement, 
showing the specific location of the TMS coil relative to the targeted brain 
region using MRI guidance. 

Patient response tracking software monitors and records the patient's responses 
to stimulation. This software collects data on various parameters, including 
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motor-evoked potentials (MEPs) and other physiological responses. The data 
is analyzed to assess the treatment's effectiveness and make any necessary 
adjustments to the stimulation parameters. This continuous feedback loop 
helps optimize the therapy for individual patients. These software tools ensure 
that TMS treatments are administered accurately, safely, and effectively, 
tailored to each patient's needs[168]. 


Advantages and Disadvantages of TMS 


The table below provides a detailed comparison of the advantages and 
disadvantages of TMS [169]: 


Disadvantages 


Limited penetration depth 


The magnetic fields generated by 
the TMS coil can only penetrate a 
few centimeters into the brain, 
limiting the treatment to 
superficial cortical areas and 
making it less effective for deeper 
brain structures. 


Advantages 


Noninvasive, outpatient 
treatment 


TMS does not require surgery or 
anesthesia, making it a_ less 
invasive option than other 
neurological treatments like Deep 
Brain Stimulation (DBS). Patients 
can receive TMS treatment in an 
outpatient setting without 
hospitalization, allowing them to 
resume daily activities 
immediately after the session. 


Eucietoranunalerecaltion a multiple sessions for 


TMS allows for precise timing of 
stimulation, which is crucial for 
understanding and modulating 
brain functions that occur on a 
millisecond scale. This high 
temporal resolution enables 
researchers and clinicians to study 


Effective treatment with TMS 
often requires a series of sessions 
over several weeks. This can be 
time-consuming and may require 
significant commitment from 
patients. Additionally, the 
cumulative cost of multiple 
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the brain's dynamic processes and 
apply targeted therapies. 
as : Variable response among 
Minimal side effects : 
patients 


Compared to pharmacological 
treatments, TMS has fewer side 
effects, the most common being 
mild scalp discomfort — or 
headache, which typically 
resolves shortly after the session. 


sessions can be a financial 


burden. 


The efficacy of TMS can vary 
significantly among patients, 
with some experiencing 
substantial benefits while others 
may see minimal improvement. 
This variability can make it 
challenging to predict treatment 
outcomes. 


Ability to target specific brain Damitedilanpereemtciaes 
regions 


TMS can be precisely targeted to 


specific brain regions involved in 


various neurological and 
psychiatric disorders, allowing for 
focused treatment that minimizes 
impact on other brain areas. 


While TMS has shown promise 
in treating certain conditions, 
there is still limited long-term 
data on its efficacy and safety, 
particularly for newer 
indications. Further research is 
needed to understand the long- 
term effects of TMS fully. 


Non-pharmacological Initial setup and calibration 
treatment option required 


TMS provides an alternative to 
medication, which is particularly 
beneficial for patients who do not 
respond well to drugs or 
experience adverse effects from 
them. 
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The initial setup and calibration of 
the TMS machine _ require 
specialized knowledge and can be 
time-consuming. Proper coil 
positioning and adjustment of 
stimulation parameters are crucial 
for effective treatment. 


Deep Brain Stimulation (DBS) 


Deep Brain Stimulation (DBS) is an invasive neuromodulation technique that 


involves implanting electrodes into specific brain regions to modulate neural 


activity. This section explores the mechanism behind DBS using the provided 


image. 


DBS Mechanism 


DBS delivers continuous electrical impulses to targeted areas in the brain 


through implanted electrodes. These impulses modulate neural activity, 


providing therapeutic benefits for neurological and psychiatric disorders [167]. 


Fig. 7.5 [174] illustrates the complex mechanism of action of DBS: 


1. 


Entrainment of Axonal Orthodromic Action Potentials (APs): DBS 
electrodes deliver electrical pulses that entrain axonal orthodromic 
APs, leading to consistent and patterned neural activity. 

Antidromic APs Collide with Intrinsic Orthodromic APs: The 
electrical stimulation generates antidromic APs, which travel 
backward along the axon. These antidromic APs can collide with the 
naturally occurring orthodromic APs, altering the overall activity of 
the neuron. 

Excitation of Afferent Inhibitory and Excitatory Fibers Projecting to 
Target Neurons: The stimulation excites inhibitory and excitatory 
afferent fibers that project to the target neurons, modulating their 
activity. 

Excitation of Passing Fibers Projecting to the Targets: DBS can also 
affect passing fibers that project to other targets, influencing neural 
circuits beyond the immediate vicinity of the electrode. 
Neurotransmitter Release: The electrical impulses stimulate 
neurotransmitter release, which can enhance or inhibit synaptic 
transmission, depending on the neurotransmitters involved. 
Microenvironmental Effects on Non-Neuronal Cells: DBS impacts the 
microenvironment, affecting non-neuronal cells such as astrocytes and 
microglia, which play critical roles in maintaining neural health and 
modulating synaptic activity. 
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7. Effects on Blood-Brain Barrier: The stimulation can influence the 
permeability of the blood-brain barrier, potentially affecting the 
movement of molecules between the bloodstream and the brain. 

DBS provides therapeutic benefits by precisely modulating neural circuits 
involved in various disorders. Understanding these mechanisms helps 
optimize DBS settings and improve patient outcomes. 
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Figure 7.5: The mechanism of action of DBS, showing the electrode delivering elec- 
trical stimulation and its effects on neuronal and non-neuronal cells. [174] 


DBS Hardware 


DBS systems have several key components that deliver precise electrical 
stimulation to targeted brain regions. The main components include electrodes, 
lead wires, implantable pulse generators (IPGs), and extension cables (Fig. 7.6 
[175]). 
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The electrodes are designed to be implanted in specific brain regions 
associated with the disorder being treated. They are typically made of 
biocompatible materials and feature multiple contacts that allow for precise 
stimulation of target areas [169]. The IPG, also known as the neurostimulator, 
is implanted in the chest or abdomen and generates the electrical pulses 
delivered to the brain through the electrodes. The IPG can be programmed to 
adjust the stimulation's frequency, intensity, and duration to optimize 
therapeutic outcomes [173]. 

The setup and implantation of a DBS system involve a multi-step surgical 
procedure. Initially, a neurosurgeon creates a small opening in the skull to 
insert the electrodes into the targeted brain region. The exact placement is 
guided by advanced imaging techniques and neurophysiological mapping to 
ensure precision. After the electrodes are implanted, the lead wires are 
tunneled under the skin to connect with the IPG, which is placed in a 
subcutaneous pocket in the chest or abdomen. Following the surgery, the IPG 
is programmed and adjusted based on the patient's response to the stimulation. 
Regular follow-up visits are necessary to fine-tune the settings and ensure 
optimal therapeutic benefits. 

By understanding DBS systems' components and setup process, clinicians can 
effectively utilize this technology to treat various neurological and psychiatric 
disorders, enhancing patient outcomes and quality of life. 


DBS Software 


DBS software plays a crucial role in managing and optimizing DBS therapy. 
The software includes programming tools, remote monitoring and 
adjustments, and data management systems [173]. 
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The programming tools in DBS software are essential for customizing the 
stimulation parameters, ensuring the therapy is tailored to each patient's 
specific needs. Clinicians can adjust the frequency, intensity, and duration of 
the electrical pulses and the configuration of the electrode contacts. This level 
of customization is critical for achieving optimal therapeutic outcomes and 
minimizing side effects, allowing for precise targeting of the affected brain 
areas [173]. 

Remote monitoring and adjustments are another critical feature of DBS 
software. Clinicians can remotely access the DBS system to monitor the 
patient's status and make necessary real-time adjustments. This capability 
enhances the therapy's flexibility and responsiveness, ensuring patients receive 
continuous and effective treatment. It also allows for timely interventions 


i ELECTRODE 


Ne 


b 


Figure 7.6: Components of a DBS system, including the electrode, lead wire, extension, and 
neurostimulator. [175] 
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without the need for frequent in-person visits, improving the overall 
management of the therapy. 

The software includes robust data management tools for recording and 
analyzing patient data. It tracks various parameters, including the settings of 
the stimulation, the patient's physiological responses, and any adverse effects. 
This data is crucial for evaluating the effectiveness of the therapy and making 
informed decisions about adjustments. Additionally, the software incorporates 
safety protocols and error detection features to ensure the system operates 
within safe limits and promptly identifies potential issues. By integrating these 
advanced data management capabilities, DBS therapy can be effectively 
monitored and optimized, providing significant benefits for patients with 
neurological and psychiatric disorders [173]. 


Advantages and Disadvantages of DBS 


The table below provides a detailed comparison of the advantages and 
disadvantages of Deep Brain Stimulation (DBS): 


Advantages Disadvantages 


Effective for treatment- Invasive with potential surgical 
resistant disorders risks 


DBS is highly effective for The procedure involves surgery, 
disorders that do not respond to which carries inherent risks such 
conventional treatments, as infection, bleeding, or adverse 
providing relief where other reactions to anesthesia. 


therapies have failed. 


Direct stimulation allows for High cost and need for periodic 
targeted treatment. battery replacements. 


High cost and need for periodic The high cost of the DBS system 

battery replacements and the need for periodic 
replacements of the battery in the 
neurostimulator can be financially 
burdensome for patients. 
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Adjustable and reversible Potential for hardware 
therapy complications 


The stimulation parameters can There is a risk of hardware-related 
be adjusted to optimize issues such as lead migration, 
therapeutic outcomes and device malfunction, or breakage, 
minimize side effects, and the which may require additional 
system can be turned off or surgeries to correct. 

removed if necessary. 


Improves quality of life and Possible side effects and 
daily functioning complications 


Many _spatients experience Potential side effects include 
significant improvements in speech problems, balance issues, 
symptoms, leading to _ better or changes in mood or behavior, 
quality of life and increased which may require adjustments to 
ability to perform daily the stimulation settings. 

activities. 


Potential for reduced Long recovery and adjustment 
medication use period 


DBS can reduce the need for Recovery from surgery and 
medications, thereby decreasing adjustment to the DBS system can 
the side effects and_ risks be lengthy and may require several 
associated with long-term drug visits for tuning and optimization. 
use. 


Current State of the Art 


Innovations in Neurostimulation 


Recent advancements in neurostimulation have significantly enhanced the 
efficacy and safety of treatments like TMS and DBS. One of the latest 
developments is the improvement in electrode technology, which has led to 
more precise and effective stimulation with fewer side effects. Advanced 
materials and designs enable better targeting of specific brain regions, 
improving therapeutic outcomes. 
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Software algorithms have also substantially improved, providing more 
sophisticated programming tools for highly individualized treatment plans. 
These algorithms can adjust stimulation parameters in real-time, optimizing 
therapy based on patient responses. Integrating artificial intelligence (AI) and 
real-time monitoring systems further enhances the ability to tailor treatments, 
providing continuous feedback and adjustments that improve overall 
effectiveness. 

Another exciting innovation is the development of noninvasive DBS 
techniques. While traditional DBS requires the surgical implantation of 
electrodes, research explores methods to deliver similar therapeutic benefits 
without invasive procedures. These noninvasive approaches could reduce the 
risks associated with surgery and make neurostimulation therapies accessible 
to a broader range of patients [176]. 


Challenges and Future Directions 


Despite these advancements, several challenges remain in the field of 
neurostimulation. Ethical considerations are paramount, particularly 
conceming the long-term effects of brain stimulation and the potential for 
misuse. Ensuring patient safety and informed consent is critical as these 
technologies evolve and are widely used. 

New neurostimulation applications in treating neurological and psychiatric 
disorders also have significant potential. Ongoing research explores how these 
technologies can be adapted to address a broader range of conditions, including 
those currently challenging to treat with existing therapies [177]. This includes 
investigating the underlying mechanisms of neurostimulation to understand 
better how it affects brain function and developing more effective and targeted 
treatments. 


Chapter 7: Summary 


Neurostimulation technologies like TMS and DBS have had a profound impact 
on modern medicine, offering new hope for patients with treatment-resistant 
neurological and psychiatric disorders. These therapies provide targeted, 
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effective treatments with fewer side effects than traditional methods, 
improving the quality of life for many individuals. 

The importance of ongoing research and development in this field cannot be 
overstated. Continued advancements in electrode technology, software 
algorithms, and non-invasive techniques will enhance the efficacy and safety 
of neurostimulation therapies. Additionally, addressing ethical considerations 
and expanding the range of treatable conditions will be crucial for the future 
of neurostimulation. By fostering innovation and rigorous research, the 
potential of these life-changing technologies can be fully realized, offering 
better outcomes for patients worldwide. 
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Chapter 7: Learning Activities 


Learning Activity 7.1 


Learning Activity: Create a Concept Map for 
a TMS Machine 


Objective 


Students will individually and collaboratively create a concept map connecting 
and explaining a TMS machine's various parts and functions. 

A concept map is a visual representation tool used to organize and structure 
knowledge by displaying relationships among concepts in a graphical format. 
It typically consists of nodes, points, or circles representing individual 
concepts or ideas, and links, which are lines connecting the nodes to illustrate 
relationships. These links often include words or phrases specifying the nature 
of the relationship, such as "causes," "includes," or "is part of." Concept maps 
usually have a hierarchical structure, with the most general concepts at the top 
and more specific details below, highlighting levels of abstraction and the flow 
of information. Key features of concept maps include a central concept, 
branches for subtopics or related concepts, and cross-links that show 
relationships between different segments, helping to illustrate interconnected 
knowledge areas. Concept maps are useful for organizing information, aiding 
learning and teaching by breaking down complex information into manageable 
parts, facilitating brainstorming, assisting in problem-solving by identifying 
relationships and dependencies, and planning projects by outlining key 
components and their relationships. For example, a concept map for a TMS 
machine might have "TMS Machine" as the central node, with branches for 
components like the coil, pulse generator, cooling system, user interface, and 
power supply, each with sub-nodes detailing specific attributes and functions. 
This structured, visual approach makes concept maps an effective tool for 
understanding and remembering complex topics. 


Materials Needed 


e Large sheets of paper or whiteboard 
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e Markers or pens 

e Sticky notes (optional) 

e Reference materials on TMS machines (textbooks, diagrams, online 
resources) 

e Concept mapping software (optional, e.g., MindMeister, Coggle) 


Preparation (Before Class) 


Assignment: Students are assigned to research the TMS machine, its 
components, and their functions before class. They should come prepared with 
notes and basic understanding. 


Stage 1: Individual Work (10 minutes) 


1. Individual Concept Map Creation: 
o Work Time (10 minutes): Students work individually to 
draft their concept maps. They start with the central node 
(TMS machine) and create branches for each main 
component, adding sub-branches and connections as they see 
fit based on their pre-class research. 


Stage 2: Collaborative Work 
Step 1: Pairs (10 minutes) 
1. Form Pairs: 
e Pair Up: Students pair up to compare their concept maps. 
2. Combine and Improve: 

e Discussion (5 minutes): Each pair discusses their maps, noting 
similarities and differences. 

e Merged Map (5 minutes): Together, they create a combined 
concept map that incorporates the best elements from both 
individual maps, ensuring all components are accurately 
represented and well-connected. 


Step 2: Groups of Three (15 minutes) 


1. Form Groups of Three: 
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o Combine Pairs: Each pair joins another student or another 

pair to form a group of three. 
2. Refine and Expand: 

o Discussion (5 minutes): Each group member presents their 
combined maps, discussing key points and connections. 

o Unified Map (10 minutes): The group creates a final, 
unified concept map that integrates input from all members. 
They refine connections, add missing details, and ensure 
clarity and accuracy. 


Debrief to the Class (15 minutes) 


1. Group Presentations: 

o Presentation (2 minutes per group): Each group presents 
their concept map to the class, explaining their connections 
and how they decided on the final structure. 

2. Class Discussion: 

o Feedback (3 minutes): After each presentation, allow time 

for questions and feedback from peers and the instructor. 
3. Reflection: 

o Facilitated Discussion (5 minutes): Engage the class ina 
discussion about what they learned from the activity. Ask 
students how working individually and then collaboratively 
helped them understand the TMS machine better. 

o Key Takeaways: Summarize the main points and correct 
any misconceptions. 


Assessment 


e Completeness: Ensure all key components and their relationships 
are included. 

e Accuracy: Verify that the information and connections accurately 
represent the TMS machine. 

e Clarity: Evaluate the clarity and readability of the concept maps. 

e Collaboration: Observe the effectiveness of collaboration in pairs 
and groups of three. 

e Presentation: Assess the effectiveness of each group's presentation 
and their ability to explain their concept map. 
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Exit Ticket: 


e Individual Reflection: Before leaving, each student completes an 
exit ticket that includes the following questions: 

1. Key Insight: What is one new thing you learned about the 
TMS machine today? 

2. Collaboration Benefit: How did collaborating with your 
peers enhance your understanding of the TMS machine? 

3. Personal Contribution: What was your most significant 
contribution to your group’s concept map? 

4. Remaining Questions: Do you have any remaining 
questions or areas of confusion about the TMS machine? 


By including an exit ticket, students can reflect on their learning, provide feed- 
back on the collaborative process, and identify areas where they might need 
further clarification. Learning activity 7.2 


Learning Activity 7.2 


Understanding Deep Brain Stimulation 
(DBS) 


Objective 


Students will enhance their understanding of Deep Brain Stimulation (DBS) 
by watching an informative video, participating in a collaborative discussion 
using the "yes, and..." technique, and ensuring every student has the 
opportunity to speak. 


Materials Needed 


e Projector and computer to show the video 
e Internet access to watch the YouTube video 
e Whiteboard and markers 

e Timer 
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Activity Steps 


1. Watch the Video (10 minutes): 


©) 


(0) 


Introduction: Briefly introduce the topic of Deep Brain 
Stimulation and its importance in medical science. 
Video Viewing: Watch the YouTube video "Deep Brain 
Stimulation" in class. 

« Deep Brain Stimulation Video 


2. Collaborative Discussion: ''Yes, and..."" Technique (30 
minutes): 


(9) 


(0) 


Ne 


(2) 


Explain the Technique: Briefly explain the "yes, and..." 
technique. Each student should build on the previous 
student's comment by responding with "yes, and...". This 
promotes active listening and collaborative thinking. 
Discussion Topics: 


What impressed you the most about DBS? 
Which parameters can you change in DBS stimulation? 


Participation Rule: Introduce the "1/x" rule, where "x" is 
the number of students in the class. This means each student 
gets approximately 1/x of the total discussion time, ensuring 
everyone has a chance to speak. 


3. Discussion Facilitation: 


(e) 


Assign Roles: Designate a discussion facilitator to keep 
track of time and ensure everyone follows the "yes, and..." 
technique. The facilitator can also prompt students who 
haven't spoken yet. 

Round-Robin Format: Start the discussion in a round-robin 
format to ensure each student has an equal chance to speak. 
Encourage students to build on each other's comments using 
"ves, and...". 

Whiteboard Notes: Use the whiteboard to jot down key 
points from the discussion, highlighting the most impressive 
aspects of DBS and the parameters that can be adjusted. 


4. Debrief and Reflection (10 minutes): 


(0) 


Summary: Summarize the main points discussed, 
highlighting what impressed students the most and the key 
parameters of DBS that can be changed. 

Reflection Questions: Ask students to reflect on the 
discussion process and their learning: 
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«What new insights did you gain about DBS? 
" How did the "yes, and..." technique affect the 
discussion? 
» What did you learn from your classmates' 
contributions? 
5. Exit Ticket (5 minutes): 
o Individual Reflection: Each student completes an exit ticket 
answering the following questions: 
1. What is one new thing you learned about DBS 
today? 
2.How did the "yes, and..." technique help you in the 
discussion? 
3. What is one question you still have about DBS? 


Assessment: 


e Participation: Ensure each student contributes to the discussion 
using the "yes, and..." technique. 

e Understanding: Evaluate students' knowledge of DBS based on 
their contributions and exit tickets. 

e Collaboration: Assess how well students build on each other's ideas 
and work together to explore the topic. Learning activity 7.3 


Learning Activity 7.3 


Exploring TMS vs. BCI through 
Quescussion 


Objective 


Students will critically evaluate and discuss the merits and drawbacks of 
Transcranial Magnetic Stimulation (TMS) and Brain-Computer Interfaces 
(BCI) through structured Quescussion. 


Materials Needed: 


e Whiteboard and markers 
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e Timer 


Activity Steps: 
1. Introduction (5 minutes): 

e Context Setting: Briefly introduce TMS and BCI, highlighting 
their primary uses and differences. Explain that today’s activity 
will involve a Quescussion—a type of discussion where only 
questions are allowed. 

2. Edgy Statement (5 minutes): 

e Present the Statement: Write the following on the board: "TMS 
is better than BCI." 

e Explain Quescussion: Inform students that they can only respond 
to the statement with questions. The goal is to explore the topic 
deeply through inquiry, challenging assumptions, and uncovering 
different perspectives. 

3. Quescussion Activity (15 minutes): 
e Rules and Roles: 
o No statements, only questions are allowed. 
o Encourage every student to participate. Use a timer to en- 
sure equal participation. 
o Designate a scribe to write down key questions on the 
board. 
Begin the Quescussion: 
o Ifneeded, prompt with an initial question: "What makes TMS 
better than BCI?" 
o Allow students to build on each other’s questions, maintaining 
the inquiry flow. 
4. Facilitation and Participation: 

e Monitor Participation: Ensure that all students are asking ques- 
tions and that the discussion remains focused. Use the 1/x rule to 
remind students that everyone should have equal time to contrib- 
ute. 
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e Encourage Depth: Prompt students to ask deeper, more probing 


questions: "How do the long-term effects of TMS compare to 
those of BCI?" or "What specific applications of TMS might be 
considered superior to BCI?" 

Transition to Open Discussion (10 minutes): 

e End of Quescussion: After 15 minutes, transition to an open dis- 


cussion. Allow students to start making statements and building 


on the questions raised during the Quescussion. 
e Discussion Points: 


oO 
12) 


What were the most compelling questions raised? 

How do the advantages and disadvantages of TMS and 
BCI compare? 

Which scenarios or applications might make one technol- 
ogy preferable over the other? 


Debrief and Reflection (10 minutes): 
e Summary of Key Points: Summarize the main questions and in- 


sights from the Quescussion and subsequent discussion. 
e Reflective Questions: 


[e) 


What did you learn about TMS and BCI that you didn’t 
know before? 

How did the Quescussion format influence your thinking 
about the topic? 

Do you feel more informed about the strengths and weak- 
nesses of TMS and BCI? 


Exit Ticket (5 minutes): 
e Individual Reflection: Each student completes an exit ticket an- 


swering the following questions: 


1. What is one significant question you have about TMS or BCI 


after today’s activity? 


2. How did the Quescussion format help you understand the 


topic better? 


3. Based on today’s discussion, which technology do you think 


holds more promise for the future, TMS or BCI, and why? 
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Assessment: 


Participation: Ensure each student asks questions and engages in 
the Quescussion and open discussion. 

Critical Thinking: Evaluate the depth and relevance of the ques- 
tions asked during the Quescussion. 

Understanding: Assess students' understanding of TMS and BCI 
based on their contributions and exit tickets. 

Reflection: Review exit tickets to gauge individual reflections 
and lingering questions about TMS and BCI. 


@ 
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Chapter 7: Lab introduction 


In this series of lab exercises, you will explore the effects of different 
forms of brain stimulation through Python simulations. These labs 
are designed to provide practical experience in understanding how 
various stimulation techniques influence neural activity. 

You will begin by simulating Transcranial Magnetic Stimulation 
(TMS) to understand its mechanisms and effects. In this lab, you will 
use Python to model the magnetic field distribution generated by a 
TMS coil and analyze the electric currents induced in cortical 
neurons. This exercise will help you grasp how TMS influences 
brain activity and its potential applications in neuroengineering and 
neuroscience research. 

In the subsequent lab, you will simulate the effects of electrical 
stimulation on a neural network, creating a graphical user interface 
(GUI) for interactive experimentation. By adjusting different 


parameters and observing their impact on neural activity, you will 


gain insights into the dynamics of neural networks and the effects of 
various stimulation parameters. This hands-on approach will deepen 
your understanding of how electrical stimulation can modify neural 
responses and behavior. 
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Chapter 7: Lab Example 1 


Laboratory Exercise: Exploring TMS Mech- 
anisms and Effects Using Python 


Objective 


This exercise aims to help students understand the mechanisms and effects of 
Transcranial Magnetic Stimulation (TMS) through simulation and analysis 
using Python. Students will simulate the magnetic field distribution generated 
by a TMS coil and analyze the induced electric currents in cortical neurons. 


Materials Needed 


- Python programming environment (e.g., Jupyter Notebook) 
- Python libraries: NumPy, Matplotlib, SciPy 


Instructions 


1. Setting Up the Environment: 
Install the necessary Python libraries if not already installed. 


!pip install numpy matplotlib scipy 


2. Simulating the TMS Coil Magnetic Field: 
The following code simulates the magnetic field generated by a 
figure-eight TMS coil. 


import numpy as np 
import matplotlib. pyplot as plt 
from scipy.constants import mu_0 


Cleve iMepesice Slo, Wy, COs wechus0. 0s, Clie 
rent=1.0): 
Magnetic field calculation for a figure- 

eight coil 

Bx = Oe current score radius 7/ (25 ( (2 — 
Coil Trad tie ye ty) 3/2)) 
By = mu) (‘current “coll radius2 / (2 (x2 + 
(y= coil radiis)iZ)(372)) 
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return Bx, By 


Create a grid of points 
x = np.linspace(-0.1, 0.1, 400) 
VES Omens aecen(S0 aly m0 lem: O10)) 
X, Y = np.meshgrid(x, y) 


Calculate the magnetic field components 
Bx, By = magnetic field(x, Y) 


Plot the magnetic field distribution 

plt.figure(figsize=(8, 6)) 

llc Surscmoloc(<, W, Bx, By, Coloma. Semec (Eee 
By2), linewidth=1.5) 

plt.title('Magnetic Field Distribution of TMS 
(@aL1L")) 

plt.xlabel('x (meters) ') 

plt.ylabel('y (meters) ') 

plt.colorbar(label='Magnetic Field Strength 
(Tesla) ') 

plit.show() 


3. Simulating Induced Electric Currents: 


The induced electric currents in the cortical neurons can be simulated using 
the curl of the magnetic field. 


det electric ticld(ex, By, dx): 

Calculate the electric field from the curl 
of the magnetic field 
Ex = np.gradient (By, dx, axis=1) 
Ey = -np.gradient (Bx, dx, axis=0) 
return Ex, Ey 


Calculate the electric field components 
Che = SL iL]| = se [{0))| 
x, Ey S Gileeicicie ieeSilel(es<, iy, Gls) 
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Pion wie Ellkecieiahe icaeSikel Clisieiciloieiom 
plt.figure(figsize=(8, 6)) 
joule .SteeSaimMollor (OK, W, les, iy, Coloma Seer (se 4p 
Ey2), linewidth=1.5) 
pilte title (tinduced Blecktrie Freld Distribution") 
plt.xlabel('x (meters) ') 
plt.ylabel('y (meters) ') 
plt.colorbar(label='Electric Field Strength 
(CW fia) 7) 
plt.show() 


4, Analyzing the Results: 


Magnetic Field Distribution of TMS Coil 
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x (meters) 


Figure 7.7: Laboratory Exercise Result 1: Magnetic Field Distribution 


324 


Induced Electric Field Distribution 
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Figure 7.8 Laboratory Exericse Result 2: Electric Field Disbtribution 


In this lab exercise, the simulation results are visualized in Figures 7.7 and 7.8, 
which depict the magnetic field distribution generated by a TMS coil and the 
corresponding induced electric field in cortical neurons. Figure 7.7 illustrates 
the magnetic field distribution, showing that the field is concentrated around 
the coil, with the strongest intensity near the center and gradually decreasing 
with distance. Figure 7.8 presents the induced electric field, which is derived 
from the curl of the magnetic field. The patterns in the electric field reflect the 
design of the coil, with the distribution playing a critical role in neuronal acti- 
vation. The variations in the electric field's intensity and focality are key fac- 
tors that can influence the effectiveness and outcomes of TMS therapy. 
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Extension: 


For advanced students, modify the simulation to explore different coil 
configurations (e.g., circular coil, double cone coil) and their effects on the 
magnetic and electric field distributions. Additionally, it simulates the effects 
of varying stimulation parameters, such as frequency and intensity, on the 
induced electric fields. 

By the end of this exercise, students should better understand the physical 
principles underlying TMS and how these principles translate into neuronal 
activation, contributing to the therapeutic effects observed in clinical settings. 


rt 
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Chapter 7: Lab Example 2 


Laboratory Exercise: Exploring Electrical 
Stimulation on Neural Networks Using Py- 
thon 


Overview 


In this lab example, we will simulate the effects of electrical stimulation on a 
neural network using Python. We will create a graphical user interface (GUI) 
to allow for interactive experimentation with different parameters and observe 
how they affect neural activity. This hands-on approach will help students 
understand the dynamics of neural networks and the impact of various 
parameters on neural activity. 


Goals of the Lab 


e Understand the basic structure and function of a neural network. 

e Explore how electrical stimulation affects neural network activity. 

e Learn how to create and interact with a GUI for scientific stimulations. 

e Analyze the impact of different parameters on neural network 
behavior, including the ratio of inhibitory neurons. 


Requirements 

e Python (tested with 3.8 and later) 

e Python libraries: NumPy, Matplotlib, Tkinter 
Steps 


The first thing you need to do is go to the GitHub repository for this book. You 
can find it using this link: 


https://github.com/bddupre92/Neurobook BME UND 


After you have opened the GitHub, you need to navigate to Chapter 7, Lab 
Example 2. In this lab example, you will find the code for the simulation. 
You will want to copy and paste the code into a script named NNTest.py and 
NNGUL py into separate Script Files. 
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Setting Up the Environment 

e Install the necessary Python libraries if not already installed. 

e =!pip install numpy matplotlib tkinter 

Creating the Neural Network Simulation 

e Define functions to create the neural network, apply electrical 
stimulation, and simulate neural activity. 

e Implement a function to simulate neural activity considering both 
excitatory and inhibitory neurons. 

Building the GUI 

e Create a graphical user interface using Tkinter that allows users to 
input parameters such as network size, stimulation intensity, 
number of steps, stimulation position, and the ratio of inhibitory 
neurons. 

Running the Simulation 

e Users will input the desired parameters and simulate the GUI. 

e The simulation will display the initial and final neural network 
activity using heatmaps. 


Interpretation of Results 


Initial Neural Network Activity: This shows the state of the neural 
network before stimulation, with only the neuron at the stimulation 
position activated. 

Neural Network Activity After Stimulation: Displays how the 
neural activity spreads from the initially stimulated neuron to its 
neighbors. The color intensity represents the activation level, with 
higher values indicating more activity. 


Key Observations 


Spread of Activation: The stimulation spreads from the initially 
stimulated neuron to its neighbors, creating a pattern of activation 
influenced by the simulation parameters. 

Effects of Inhibition: Inhibitory neurons can significantly alter the 
activation pattern, demonstrating how inhibition can dampen neural 
activity and create more complex patterns. 
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Suggestions for Parameters to Modify 


1. 


Network Size: 

e Impact: Increasing the network size allows for observing how the 
spread of activation scales with larger networks. 

e Output: Larger networks may show more complex activation 
patterns and a broader activity spread. 

Stimulation Intensity: 

e Impact: Higher intensity can result in stronger activation spread. 

e Output: Increased stimulation intensity can lead to more neurons 
being activated, resulting in a more widespread and intense pattern 
of activity. 

Number of Steps: 

e Impact: More steps can provide insights into the longer-term 
effects of stimulation. 

e Output: Additional steps allow observing how the activation 
evolves, potentially showing sustained or diminishing activity. 

Stimulation Position: 

e Impact: Changing the position can show the effect of stimulation 
at different network locations. 

e Output: Different stimulation positions can highlight how the 
network's structure and connectivity influence the spread of 
activation. 

Inhibitory Neurons Ratio: 

e Impact: Varying the ratio of inhibitory neurons affects the 
network's balance between excitation and inhibition. 

e Output: 

o Low Ratio (e.g., 0.01): A low ratio of inhibitory neurons 
results in minimal inhibition, allowing for a broader and 
more intense activity spread, as shown in the first image. 

o High Ratio (e.g., 0.2): A higher ratio of inhibitory 
neurons introduces significant inhibition, reducing the 
overall activity and leading to a more localized and 
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subdued activation pattern, as demonstrated in the second 
image. 


Detailed Explanation of the Images 
Image with Inhibitory Neurons Ratio at 0.01 


Observation: The neural network activity after stimulation shows a broad 
spread of activation. 


Interpretation: With a low ratio of inhibitory neurons, the inhibition is 
minimal, allowing more neurons to become activated and maintain higher 
activity levels across the network. 
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Figure 7.7: Neural Activity with Inhibitory Neurons 
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Image with Inhibitory Neurons Ratio at 0.2 


e Observation: The neural network activity after stimulation is more 
localized, with reduced overall activity. 

e Interpretation: A higher ratio of inhibitory neurons increases the 
inhibition, damping the spread of activation and resulting in fewer 
neurons being activated. The overall activity is more subdued and 
confined to a smaller region. 

By adjusting these parameters, students can explore the intricate dynamics of 
neural networks and understand the delicate balance between excitation and 
inhibition. This hands-on lab provides valuable insights into the behavior of 
neural systems and the potential implications for neurostimulation therapies. 


A 
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Chapter 8 


Visual Voyage: Exploring the 
Wonders of the Visual System 
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Introduction and Learning Objectives 


So far, in your journey to discover neural engineering concepts, you have 
learned about the fundamental building blocks of the brain, ways to measure 
brain activity, and how to use that activity to impact society. Now, we will 
understand a captivating part of the nervous system: the visual system. The 
visual system is a network of structures working together to capture, process, 
and interpret light. In this chapter, we will go on another journey and discuss 
the anatomy of the visual system and how it works. This chapter will give you 
an overview of the visual system and its function. By the end of this chapter, 
you will be able to: 


. Describe the anatomical features of the eye. 
. Explain the roles of the different parts of the visual system. 


. Understand the process of light interpretation and its importance in 
vision. 


Anatomy of the Eye 
The Layers of the Eye 


The human eye is a complex organ that is responsible for vision. The eye 
captures light stimuli from the environment and converts these into neural 
signals, which are then interpreted by the brain. The eye has three main layers: 
the outer, middle, and inner. The outer layer is made of the cornea and the 
sclera. The corneas are transparent structures that funnel light into the eye. The 
sclera is the white part of the eye, and its function is to provide protection and 
form to the eye. The middle layer included the choroid, ciliary body, and iris. 
The choroid contains blood vessels that supply the eye with much-needed 
nourishment. The ciliary body controls the shape of the lens, and the iris is a 
muscle that contracts and relaxes to control how much light is allowed to enter 
the eye by adjusting the size of the pupil. The lens, located behind the iris and 
pupil, focuses light onto the retina and is held in place by the suspensory 
ligament. The inner layer includes the retina, which contains several layers that 
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include photoreceptor cells, bipolar neurons, and ganglion cells. The layers of 
the retina function to detect light and convert it into electrical signals that are 
then sent to the brain through the optic nerve [178]. 


The Chambers of the Eye 


The eye can also be divided into three main chambers. These chambers are 
filled with fluid to maintain the shape of the eye and provide nourishment. The 


Anterior chamber Cornea 
(aqueous humour) 
Posterior chamber 


Suspensory 
ligament 
of lens ; 
Choroid 
Sclera 


Optic nerve Optic disc 


Figure 8.1: Anatomy of the Eye [180] 
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anterior chamber lies between the cornea and the iris and is filled with aqueous 
humor. Aqueous humor is a clear fluid that helps maintain intraocular pressure. 
The posterior chamber is between the iris and the lens and contains aqueous 
humor. This chamber plays a role in maintaining intraocular pressure and nu- 
trient transport. The largest chamber is called the vitreous chamber and is be- 
hind the lens. This chamber is filled with vitreous humor, a gel-like substance 
that helps maintain the eye’s shape. The hyaloid canal runs through this cham- 
ber from the lens to the optic disc to supply blood from the hyaloid artery to 
the lens. The macula is at the back of this chamber, and its center is called the 
fovea. The fovea is where eyesight is sharpest. [179]. See Fig. 8.1 [180] for a 
labeled graphic of the anatomy of the eye. 


Light Refraction and Phototransduction 


Vision begins with light refraction as it enters the eye and ends when electoral 
signals are sent to the brain for image processing. This process ensures accu- 
rate interpretation of visual information, providing clear and detailed vision. 
The cornea and the lens play a critical role in refraction. The retina, equipped 
with photoreceptor cells (rods and cones), converts light into neural signaling 
through phototransduction. 


Refraction 


Refraction focuses on the light that enters the eye and hits the retina. When 
light passes through the chambers and different mediums, its speed changes, 
making it bend and refract. Refraction begins at the cornea, contributing to 
most of the eye's refractive power. Light reaches the lens after light passes 
through the cornea and aqueous humor. The lens provides fine adjustment to 
the focus of light through its ability to change shape, known as accommodation 
(Fig. 8.2 [181]). The ciliary muscle and suspensory ligaments flatten the lens 
for distant vision and round it for near vision. This control ensures the light 
converges accurately on the retina, making images clear and focused. 


335 


Vitreous body 


Retina 


Aqueous humor 
(b) 


Figure 8.2: Refraction of Light in the Eye by the Lens [181] 


The Retina and its photoreceptors 


After light is channeled through these structures, the retina will convert the 
light into neural signals. The retina can do this through photoreceptor cells, 


Epithelial 
cells 


Figure 8.3: Retina layers responsible for phototransduction 1) light enters the eye and refracts 
on the retina 2) light passes to the outermost layer of the retina to the photoreceptors 3) bipolar 
cells mitigate and translate the signal to the ganglion cells 4) ganglion cells combine to make 
up the optic nerve 5) optic nerve sends the signal to the brain [182] 
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Figure 8.4: Diagram of a Rod and Cone [182] 


rods, and cones (Fig. 8.3 [182]). The rods are highly light-sensitive and allow 
vision in low-light conditions. The rods cannot detect color and are concen- 
trated around the retina's periphery. The cones are less light-sensitive but are 
responsible for color vision and visual acuity. The cones are concentrated in 
the fovea. As mentioned earlier, the retina also contains two other layers. The 
retina contains different cell types in these layers, such as bipolar and ganglion 
cells. These cells process visual information before transmitting it to the brain 
[183]. All the parts of the retina are crucial for phototransduction. 


Phototransduction 


Phototransduction begins when photopigments on the photoreceptors absorb 
light. This happens when light passes through the layers of the retina to reach 
the back layer that contains the photoreceptors. In rods, the photopigment is 
rhodopsin. The cones contain three photopigments corresponding to light 
wavelengths: red, green, and blue. When light activates these photopigments, 
it triggers a series of biochemical events that lead to hyperpolarization of the 
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photoreceptor membrane. The change in membrane potential causes the 
release of neurotransmitters (Chapter 1) that synapse with a bipolar cell. The 
bipolar cells modulate that signal and relay it to ganglion cells and the optic 
nerve. The optic nerve then carries this signal to the brain. This process ensures 
that light information is converted into neural signals the brain can interpret 
[6], [183]. Fig. 8.4 [182] shows a detailed image of light travels through the 
eye and into the layers of the retina to transform light stimuli into electrical 
signals to be sent to the brain. 


Visual Pathway 


The visual pathway is a complex network that begins at the retina and extends 
to the visual cortex in the brain. In this section, we will explore the signals that 
signals take to the brain from the eye, the role of the visual cortex, and how 


noropter 


Figure 8.5: Binocular Vision [184] 
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this information is integrated and perceived to result in the visual experiences 
we encounter daily. 


The Journey to the Brain 


The visual information from the retina undergoes a journey to reach the visual 
cortex. After leaving the eye, the optic nerve travels through the optic chiasm, 
where the nerves partially cross—the nerve fibers stemming from each 
retinal's nasal half cross to the brain's opposite side. The fibers from the 
temporal halves of the retina do not cross and remain on the same side. This 
crossing ensures that both hemispheres of the brain process visual information 
from each eye to create binocular vision and depth perception. Binocular 
vision is our ability to perceive a single, cohesive visual field from the input 
of both eyes, allowing for a three-dimensional understanding of our 
environment (Fig. 8.5 [184]). The overlapping fields of view from each eye 
provide the brain with slightly different perspectives, which are then integrated 
to create a single three-dimensional image. 

After the optic chiasms, the visual information continues its journey along the 
optic tracts to the later geniculate nucleus (LGN) of the Thalamus. Recall from 
Chapter 1 that the thalamus is in the diencephalon and is the relay station for 
sensory signals to higher brain areas. The LGN refines and processes the visual 
signals before sending them to the primary visual cortex in the occipital lobe 
via optic radiation. The optic radiation is divided into two pathways. The upper 
path carries information from the lower visual field, and the lower pathway 
from the upper visual field [185]. 


The Visual Cortex 


The primary visual cortex (V1) is the first cortical area to receive visual 
information from the LGN and is in the occipital lobe at the back of the brain. 
V1 is organized into a map of the visual field, called retinotopic organization. 
Retinotopic organization means neurons that pick up visual stimuli near each 
other in our visual fields also have neurons close together in the cortex. Fig. 
8.6 [186] depicts the retinotopic organization of V. The retinotopic 
organization of V1 ensures that spatial relationships in the visual field are 
preserved during processing—V1 processes vision attributes, such as edges, 
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orientation, and motion. V1 
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color perception. These areas ual Cortex [186] 


work together to integrate all the 
elements of visuals to build a more detailed and compressive representation of 
our visual field [6]. 


Integration and Perception 


The integration and perception of visual information are complex processes 
that involve the coordination of multiple specialized regions within the brain. 
These processes are crucial for basic visual recognition and higher-level cog- 
nitive functions that enable us to navigate and interact with our environment 
effectively. The visual system is organized into two primary processing 
streams, each responsible for different aspects of visual perception. 

The Dorsal Stream 

The Dorsal Steam originates in the occipital lobe, specifically the primary vis- 
ual cortex (V1), and projects to the parietal lobe. This pathway is often called 
the "where" pathway because it is primarily involved in processing spatial in- 
formation about the location and movement of objects in the visual field. The 
dorsal stream allows us to determine the position of objects relative to our own 
body, which is essential for spatial awareness. This spatial processing is critical 
for tasks such as reaching for an object, navigating through a room, or tracking 
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moving objects. In addition to spatial awareness, the dorsal stream plays a cru- 
cial role in movement coordination. It integrates visual information with motor 
commands to guide actions, such as adjusting the grip on a cup or timing a 
catch during a game. This stream is heavily involved in real-time processing, 
allowing for quick environmental adjustment. 


The Ventral Stream 


The Ventral Stream also begins in the occipital lobe but projects to the tem- 
poral lobe. It is known as the "what" pathway because it identifies and recog- 
nizes objects, faces, and scenes. The ventral stream is crucial for recognizing 
shapes, colors, and textures, which are the building blocks of object recogni- 
tion. This pathway allows for identifying everyday objects, such as recogniz- 
ing a car or distinguishing between different types of fruit. Certain areas, such 
as the fusiform face area (FFA), are specialized for recognizing faces within 
the ventral stream. This ability is vital for social interactions, allowing us to 
remember friends and family members and assess emotional expressions. The 
ventral stream also contributes to the recognition and interpretation of complex 
scenes, enabling us to understand the context of our surroundings, such as dis- 
tinguishing a busy street from a quiet park. 

Once visual information is processed through the dorsal and ventral streams, 
it is integrated with other sensory inputs and higher-order cognitive processes 
in various cortical regions. This integration is essential for creating a coherent 
and comprehensive visual perception. In higher cortical areas, visual infor- 
mation is combined with inputs from other sensory modalities, such as audi- 
tory, tactile, and proprioceptive signals. This multisensory integration allows 
for a more robust and accurate perception of the environment. For instance, 
hearing a car while seeing its approach helps to gauge its speed and distance 
more effectively than visual input alone. The brain uses context and prior 
knowledge to interpret visual information. This involves integrating visual 
data with memories, expectations, and experiences stored in the brain. For ex- 
ample, recognizing a familiar face in a crowd or identifying an object based 
on its surroundings relies on this contextual processing. Attention plays a crit- 
ical role in visual perception by selecting which visual information is pro- 
cessed more deeply and which is filtered out. Higher cortical areas, such as the 
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prefrontal cortex, interact with visual areas to direct attention to relevant stim- 
uli, such as focusing on a pedestrian while driving or finding a friend in a 
crowded room. Integrated visual information is also used to guide actions. For 
example, the brain's ability to recognize a door handle and coordinate the 
movement to grasp and turn it seamlessly involves the interaction of visual 
perception and motor planning areas. The parietal lobe plays a key role in this 
visuomotor integration through its connection with the dorsal stream. 

The brain's ability to integrate and interpret visual information rapidly and ac- 
curately is essential for interacting with our environment. This capability is 
needed for nearly every aspect of daily life, from simple tasks like reaching 
for a cup of coffee to complex activities like playing sports or driving a car. 
The speed and accuracy of visual processing are critical for survival, allowing 
for quick reactions to potential threats or opportunities in the environment. For 
example, detecting a moving vehicle in peripheral vision and responding ap- 
propriately directly results from the brain's efficient visual integration. Visual 
integration is also fundamental to learning and memory. By associating visual 
cues with experiences, the brain builds a repository of knowledge that informs 
future behavior. This process is crucial for tasks ranging from recognizing fa- 
miliar objects to navigating new environments. In social contexts, the ability 
to quickly recognize faces, interpret expressions and understand gestures relies 
on the seamless integration of visual information. This capacity enables effec- 
tive communication and helps us navigate complex social dynamics. 

The integration and perception of visual information in the brain involves a 
sophisticated network of pathways and cortical regions working together. The 
dorsal and ventral streams process visual stimuli, such as spatial location and 
object identification, which are then integrated with other sensory inputs and 
cognitive processes in higher cortical areas. This complex integration allows 
us to coherently understand our surroundings and interact with the world in 
meaningful and adaptive ways. The efficiency and accuracy of this system are 
essential for everything from basic survival to complex social interactions, 
highlighting the remarkable capabilities of the human brain. 
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Visual Disorders and Disease 


Now that we have explored how the visual system works, addressing visual 
disorders and diseases is essential. Visual disorders can significantly impact an 
individual's quality of life, ranging from minor vision impairments to complete 
blindness. Understanding these conditions is necessary for accurate diagnosis 
and for innovating new treatments. Advancements in neuroengineering offer 
promising solutions and help enhance our ability to manage and treat these 
disorders. The following section will discuss some of the most common visual 
disorders and diseases and the current geoengineering solutions for their 
treatment and management. 


Refractive Errors 


Refractive errors occur when the eye’s shape (optical imperfections) prevents 
light from focusing directly on the retina, causing blurred vision. These 
refractive errors include nearsightedness or myopia, farsightedness or 
hyperopia, astigmatism, and presbyopia. These conditions are caused by 
structural irregularities in the eye, including the eyeball's length or the cornea's 
shape. Common symptoms of refractive errors include difficulty seeing distant 
or close objects. Treatments for refractive errors include advanced engineering 
solutions such as wavefront-guided LASIK and intraocular lenses (IOLs). 
These technologies use eye measurements to customize treatments to improve 
visual outcomes. Additionally, adaptive optics are being adapted for retinal 
imaging and corrective procedures to provide vision correction [187]. 


Cataracts 


Cataracts are characterized by lens clouding, leading to decreased vision and 
even blindness. Risk factors for cataracts include age, diabetes, smoking, and 
prolonged exposure to ultraviolet light. Symptoms include blurry vision, glare, 
and difficulty seeing at night. The primary treatment for cataracts is surgical 
removal of the clouded lens and replacement with IOLs. Another engineered 
cataract treatment is femtosecond laser-assisted cataract surgery (FLACS), 
which allows for more precise and less invasive surgical procedures. 
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Additionally, advanced bioengineering lenses can improve patient visual 
outcomes and treatment safety and efficacy [188]. 


Glaucoma 


Glaucoma involves damage to the optic nerve, usually due to high intraocular 
pressure, leading to progressive vision loss. Factors that contribute to 
glaucoma include age, genetic disposition, and certain medical conditions. 
Early stages of glaucoma typically are asymptomatic, but advanced stages can 
cause peripheral vision loss. Engineering advancements have led to the 
development of minimally invasive glaucoma surgeries. These surgeries 
reduce the intraocular pressure in the eye. Additionally, devices like micro 
stents and shunts can improve eye fluid drainage. Continuous intraocular 
pressure monitoring systems that use intelligent contact lenses can also provide 
real-time data to help manage and treat the disease [189]. 


Macular Degeneration 


Macular degeneration is the deterioration of the macula, leading to loss of 
central vision. Age-related macular degeneration (AMD) is influenced by age, 
genetic doctors, and lifestyle choices. Symptoms include blurred central vision 
and difficulties in recognizing faces. Cutting-edge treatments include retinal 
implants and gene therapy. Retinal implants, or bionic eyes, can partially 
restore vision by converting light into electrical signals that stimulate the 
remaining retinal cells. Gene therapies aim to correct genetic defects that cause 
AMD. Additionally, imaging technologies like optical coherence tomography 
(OCT) provide detailed retina images, aiding in early disease detection and 
monitoring [190]. 


Diabetic Retinopathy 


Diabetic Retinopathy is a complication of diabetes that affects the blood 
vessels of the retina and leads to vision impairment. High blood sugar levels 
can damage the retinal blood vessels, causing them to swell or lose entirely. 
This can lead to vision problems. Symptoms include floaters, blurred vision, 
and dark areas of vision. Technologies to treat this condition include laser 
photocoagulation to seal leaking blood vessels to prevent further damage. 
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Vitrectomy, removing the vitreous gel and replacing it with oil, can help clear 
the blood from the eye and repair retinal detachment. Like glaucoma, 
engineered continuous glucose monitoring systems and insulin pumps can help 
maintain optimal blood sugar levels, reducing the risk of this disease. 


Retinal Detachment 


Retinal detachment happens when the retina separates from the underlying 
tissue, which can lead to vision loss if it is not treated. Causes for retinal 
detachment include trauma, severe myopia, and age-related changes. 
Symptoms include sudden flashes of light and a shadow over the field of 
vision. Advances in surgical techniques and instruments have improved the 
success of retinal detachment repairs (Fig. 8.7 [191]). Additionally, 
developments in retinal imaging and microsurgery tools have enhanced the 
precision and outcomes of these surgical procedures [192],[193]. 


Strabismus 


Strabismus, or cross eyes, is a condition where the eyes do not correctly align 
when looking at an object. This misalignment can result in double vision or 
the brain ignoring inputs from one eye to void confusion, leading to amblyopia 
or lazy eye. Causes can include muscle imbalances, nerve damage, or 
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Figure 8.7: Image of typical retinal implants with (a) outer and (b) inner components [191] 
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congenital issues. Symptoms include misaligned eyes, difficulty with depth 
perception, and double vision. Robotic-assisted surgeries are being developed 
to improve the accuracy and outcomes of surgeries used to fix the alignment 
of the eyes. Vision therapies using virtual reality (VR) and computer-based 
exercises have also been used to strengthen the eye muscles to improve 
coordination. These engineering technologies offer new ways to treat and 
manage this condition [194]. 

Understanding these disorders and diseases is crucial for improving treatment 
and management strategies. These conditions range from common refractive 
errors to more severe issues like glaucoma, which can significantly impact 
individuals’ quality of life. Integrating neuroengineering solutions into 
ophthalmology has enabled more effective treatments and better management 
of visual disorders. From laser-assisted surgeries to retinal implants, the field 
of vision care is evolving. As we move forward, continued research and 
development in neuroengineering and ophthalmology will be essential in 
addressing the challenges posed by visual disorders. 


Chapter 8: Summary 


This chapter explored the visual system's intricate world, anatomy, function, 
and various disorders affecting vision. We began with a detailed examination 
of the eye anatomy. We discussed the essential structures and their role in 
capturing and processing light and understanding how light stimulus goes 
through the eye. We then explored the visual pathway and how visual 
information travels from the retina to the brain. The discussion of retinotopic 
organization within the visual cortex shows the precision with which our brain 
processes visual stimuli. We then discuss various visual disorders, including 
their causes, symptoms, and current treatments, along with innovative 
neuroengineering solutions in the field of ophthalmology. You are encouraged 
to continue your knowledge and investment in and support for research and 
development in neuroengineering and ophthalmology. By fostering 
collaboration and investing in cutting-edge technologies, we can advance our 
understanding and treatment of visual disorders. In the next few chapters, we 
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will explore neural engineering technologies that harness vision, diving deeper 
into how these innovations are being developed and applied. 
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Chapter 8: Learning activities 


Learning Activity 8.1 


Game: let us make a trivial pursuit game. 
Create ten cards with four topics: anatomy, 
functionality, math, and graphs. 


Objective 

Students will create a section of a Trivial Pursuit game, designing question 
cards covering anatomy, functionality, math, and graphs related to 
neuroengineering. 


Materials 


e Index cards or paper 

e Pens or markers 

e Internet access (for research) 

e Reference materials (textbooks, notes) 


Time 
30-45 minutes 


Instructions 


Introduction (5 minutes): 
1. Introduce the activity and explain its goal: to create a section of a Triv- 
ial Pursuit game focused on neuroengineering topics. 
2. Divide the class into small groups of 3-4 students. 
Card Creation (25-30 minutes): 
3. Instruct each group to create 10 Trivial Pursuit cards, with questions and 
answers, covering the following topics: 
e Anatomy (e.g., brain regions, neural structures) 
e Functionality (e.g., how different parts of the brain work, 
neurochemical processes) 
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e Math (e.g., calculations related to neuroengineering, statistical 
methods) 

e Graphs (e.g., interpreting data from neuroimaging, EEG/MEG 
results) 


Guidelines for Creating Cards: 


e Each card should have a question on one side and the answer on 
the other. 

e Questions should vary in difficulty. 

e Ensure accuracy by referencing reliable sources. 

e Use a mix of question types (e.g., multiple choice, true/false, short 
answer). 


Group Sharing (5-10 minutes): 
4. Have each group share a few of their most interesting or challenging 
questions with the class. 

Conclusion (5 minutes): 


5. Collect the cards and explain that they will be used in a future Trivial 
Pursuit game session to review and reinforce learning. 6. Summarize 
the activity, emphasizing the importance of reviewing key concepts 
engagingly and interactively. 


Learning Activity 8.2 
Role-Play Game - Parts of the Eye 
Objective 


Students will learn and reinforce their knowledge of the 
anatomy and functions of different parts of the eye through an interactive role- 
play game. 
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Materials 


e List of eye parts (e.g., cornea, lens, retina, optic nerve, iris, pupil, 
sclera, vitreous humor, etc.) 

e Slips of paper with the names of different eye parts 

e A timer 

e Crown or small prize for the champion 


Time 
20-30 minutes 


Instructions 
Introduction (3 minutes): 

1. Introduce the activity and explain its goal: to reinforce knowledge of 
the anatomy and functions of the eye through a fun and interactive 
role-play game. 

2. Explain the rules of the game and how it will be played. 

Preparation (2 minutes): 

3. Randomly assign each student a part of the eye by drawing slips of paper 
with the names of different eye parts. 

4. Give students 2 minutes to prepare by reviewing their assigned part's 
anatomy and function. 


Role-Play and Questioning (15-20 minutes): 

5. Begin the rounds of questioning. One student starts by answering questions 
from their peers about their assigned eye part. 

6. If students answer correctly, they continue to the next round. They are 
eliminated if they answer incorrectly or if another student correctly guesses 
their part. 

7. Continue the rounds until only one student remains. This student is the 
champion. 


Conclusion and Crowning the Champion (5 minutes): 


8. Crown the last student standing as the champion with a small prize or a 
crown. 
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9. Summarize the key points learned during the activity and emphasize the 
importance of understanding eye anatomy and function. 


Learning Activity 8.3 
Concept Map - Parts of the Visual System 
Objective 


Students will create a concept map of the visual system to 
organize and understand its components and functions visually. This activity 
will involve individual work, peer, and group sharing to enhance collaborative 
learning. 


Materials: 
e Large sheets of paper or poster boards 
e Markers or colored pens 
e Internet access or textbooks (for reference) 
e Sticky notes (optional) 


Time: 
30-45 minutes 


Instructions: 


Introduction (5 minutes): 

1. Introduce the activity and explain its goal: to create a concept map that 
organizes and illustrates the components and functions of the visual 
system. 

2. Briefly explain what a concept map is and provide an example if nec- 
essary. 


Individual Task (10 minutes): 

3. Ask each student to create a concept map of the visual system. They 
should include key components such as the cornea, lens, retina, optic 
nerve, and other relevant parts and show how they are connected. 

4. Allow students to use reference materials like textbooks or online re- 
sources to help them. 


Peer Share (5 minutes): 
5. Once the individual maps are complete, pair up the students. 
6. Instruct each pair to share their concept maps, discuss the similarities 
and differences in their maps, and provide feedback. 


Group Share (10 minutes): 
7. Form small groups of 4-5 students and have each pair join another pair. 
8. In these groups, students will share their concept maps again, integrat- 
ing the best elements from each map to create a comprehensive group 
concept map. 
9. Provide large sheets of paper or poster boards and markers for the 
groups to draw their combined concept maps. 


Class Presentation (10-15 minutes): 
10. Each group will present their combined concept map to the class, ex- 
plaining the components and their connections. 
11. Facilitate a class discussion to compare the different maps and high- 
light key points. 


Conclusion (5 minutes): 

12. Summarize the key points learned during the activity, emphasizing the 
importance of understanding the visual system's components and their 
interconnections. 

13. Encourage students to reflect on how creating and sharing concept 
maps helped them understand the visual system better. 


@ 
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Chapter 8: Lab introduction 


In this series of lab exercises, you will explore advanced techniques 
for visual system analysis using MATLAB. These labs generate 
interactive visualizations and utilize specialized toolboxes to model 
various aspects of eye anatomy and function. 

You will begin by creating an interactive eye anatomy image using 
MATLAB. This lab will guide you through generating and 
manipulating a detailed visualization of the eye, enhancing your 
understanding of its structure and functions. Ensure that MATLAB 
is installed on your system; if not, refer to the installation instructions 
provided in Chapter 1: Lab Example 1. 

In the subsequent lab, you will work with the Image System 
Engineering Toolbox for Biology (ISETBio), a MATLAB toolbox 
designed for in-depth visual system analysis. ISETBio enables you 
to create spectral radiance scenes and model their effects on various 
components of the human visual system, including eye optics, 
movements, cone absorptions, and retinal cell properties. You will 
download and install ISETBio from the provided GitHub link and 
apply its features to explore complex visual system simulations. By 
the end of these labs, you will gain hands-on experience in 
visualizing eye anatomy and utilizing advanced tools for modeling 


visual processes. 
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Chapter 8: Lab Example 1 


Overview 

We will generate an interactive eye anatomy image using 
MATLAB in this lab. You should already have MATLAB 
installed; however, if you don’t, there are instructions on downloading it in 
Chapter 1: Lab Example 1. 


Requirements 


e MATLAB 
e Texture images of the iris, sclera, and retina 


Steps 

First, go onto the internet and find three images: one of the iris, one of the 
retina, and one of the sclera. Make sure that these images are saved a .jpg files. 
For this lab to work correctly, you must know each anatomical component and 
save the right pictures. If you need a refresher on these anatomical 
components, refer to section 9.2 of this chapter. 

After you have MATLAB open, start a new script and go to the GitHub 
repository at this link: 


https://github.com/bddupre92/Neurobook BME UND 


Navigate Chapter 8 code 1 and open it in MATLAB. 


Script Structure 
Generate Unit Sphere 

e Create a unit sphere to serve as the base for the eye model. 
Prepare Figure 

e Configure the figure properties such as color, axis, and grid. 
Main Globe (Sclera) 


e Define the radius and generate the surface for the main globe 
representing the sclera. 
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Tris Sphere 

e Define the radius and generate the surface for the iris. 

e Adjust coordinates to position the iris on the globe properly. 
Read Images 

e Load the texture images for the iris, sclera, and retina. 
Texture Mapping 

e Apply the texture images to the corresponding surfaces. 

e Optionally mirror the images to enhance the definition. 
Rotate and View Adjustment 

e Rotate the main globe for correct orientation. 

e Adjust the view to display the front of the eye and part of the back. 
Interactive Features 

e Enable interactive manipulation of the view with rotate3d. 
Retina Texture 


e Add a textured surface to represent the retina at the back of the eye. 


Key Functions Used 
sphere 


e Usage: [x, y, z] = sphere(); 
e Purpose: Generates the coordinates for a unit sphere. 


e Usage: mainGlobe = surf(xm, ym, zm, 'EdgeColor', ‘none’, 
"FaceAlpha', 0.5); 
e Purpose: Creates a 3D surface plot. 
imread 
e Usage: imglIris = imread(irisImagePath); 
e Purpose: Reads an image file into MATLAB. 
set 


e Usage: set(mainGlobe, 'FaceColor'’, 'Texturemap', 'CData', CDglobe, 
'FaceAlpha’, 0.5); 
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e Purpose: Sets properties for graphics objects. 


rotate 


e Usage: rotate(mainGlobe, [1 0 0], -90); 
e Purpose: Rotates the graphics object. 


1. Usage: view([-1, -1, 0.5]); 

2. Purpose: Sets the viewpoint for the 3D plot. 
camorbit 

e Usage: camorbit(0,0,'camera'); 

e Purpose: Orbits the camera around the scene. 
rotate3d 


e Usage: rotate3d on; 
e Purpose: Enables interactive 3D rotation of the plot. 


Further considerations 
See lines 22, 23, and 24 under 


% Define the paths to the images. 


You must make a path to your photos from your computer and paste them here. 
To create a path to an image, right-click it and select “Copy as Path.” You will 
then paste that into the MATLAB code. Make sure you have ‘’ around the file 
path name. You must ensure your images are saved in your MATLAB drive, 
where you create the path. If you do not have a MATLAB drive, follow this 
link to set it up. 


https://www.mathworks.com/products/matlab-drive.html 


Once you have entered your file paths, click run. You should see a figure 
window (Fig. 8.8) with your eyeball inside! 
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Figure 8.8: Code One Results 


We will take this further and look at it with different eyes. 
Open a new script on MATLAB and get back to GitHub. Navigate to Chapter 
8, code 2, and open it in MATLAB. 


Description of Differences in Structures and Key Functions 


This updated MATLAB script builds upon the previous one by generating and 
visualizing two 3D eyeballs with different iris textures. Here are the key 
differences and additions: 


Key Additions and Changes 
Multiple Image Paths 


The script now includes an additional path for a second iris texture image: 
irisImagePath2 = ‘iris image 2 path'; % Path to the iris 
texture image for the second eyeball 
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File Existence Checks 


The script checks if all specified image files exist, including the second iris 
image: 
if ~isfile(irisImagePath) 
error('Iris image file not found: %s', irisImagePath) ; 
end 


Reading Images 


The script reads the second iris image in addition to the other images: 
try 

imgIris = imread(irisImagePath) ; 
catch ME 

error('Error reading iris image: %s', ME.message) ; 
end 


Generating Two Eyeballs 


Coordinates and surfaces for the second eyeball are generated with a horizontal 
shift to separate them visually: 

% Define the coordinates for the second eyeball 

x_shift = 12; % Adjust as needed 

xm2 = xm + x_shift; 

xi2 = xi + x_shift; 


Surface Generation and Texture Mapping for the Second Eyeball: 


The script creates the main globe, iris, and retina surfaces for the second 


eyeball: 

% Generate the surface for the second eyeball 
mainGlobe2 = surf(xm2, ym, zm, ‘EdgeColor', ‘none’, 
"FaceAlpha', @.5); 

irisGlobe2 = surf(xi2, yi, zi, ‘EdgeColor', ‘none'); 


% Apply mapping for the second eyeball with a different iris 
image 
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set(mainGlobe2, ‘FaceColor', ‘Texturemap', 'CData', CDglobe, 
"FaceAlpha', @.5); 

set(irisGlobe2, '‘FaceColor', 'Texturemap', ‘CData', imgIris2, 
"EdgeColor', 'none'); 


% Texture mapping for the second eyeball 
rotate(mainGlobe2, [1 @ @], -90@); 


% Add retina texture to the back of the second eye 
retina2 = surf(xr + x_shift, yr, -zr, ‘FaceColor', 
"Texturemap', '‘CData', imgRetina, ‘EdgeColor', 'none'); 


Summary of Key Functions 
isfile 
e Usage: if ~isfile(irisImagePath2) 
e Purpose: Checks if a file exists at the specified path. Used to ensure 
all image files are available before proceeding. 
imread 
e Usage: imgIris2 = imread(irisImagePath2); 
e Purpose: Reads an image file into MATLAB. Extended to include 
reading the second iris image. 


e Usage: mainGlobe2 = surf(xm2, ym, zm, 'EdgeColor', 'none’, 
"FaceAlpha', 0.5); 
e Purpose: Creates a 3D surface plot. Used to generate the main globe, 
iris, and retina surfaces for both eyeballs. 
set 
e Usage: set(mainGlobe?2, 'FaceColor’, 'Texturemap', 'CData', CDglobe, 
"FaceAlpha', 0.5); 
e Purpose: Sets properties for graphics objects. Used to apply textures 
to the surfaces of both eyeballs. 
rotate 
e Usage: rotate(mainGlobe?2, [1 0 0], -90); 
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e Purpose: Rotates the graphics object. Used to orient both eyeballs 
correctly. 

Use the following images (Fig. 8.9) for irisImagePath and irisImpagePath2. 
Save them to your MATLAB drive, make an image path, and copy and paste 
them into the code. Do not forget to copy and paste your image paths for the 
sclera and the retina. 
Now, run your code. You should have a figure box with two eyes in it. If you 
receive an error code that states ‘XX image file not found,’ you have not 
created the image path correctly and saved it into your MATLAB drive. 
Can you tell me what is off with these two eyes? The right pupil is much 
smaller than the right eye pupil. Do you know what this is called? You may 
know about this disorder if you have taken a neuroanatomy class before. Hint: 
we did not discuss it in this chapter. This is called Horner's syndrome. Horner’s 
syndrome is a rare neurological syndrome that results from underlying nerve 
damage from a stroke, tumor, or spinal cord injury. Symptoms of this 
syndrome include smaller pupils (miosis), drooping eyelids (ptosis), and little 
to no sweating on the affected side of the face (anhidrosis). To treat this 
syndrome, some individuals will need the removal of a tumor or appropriate 
medical treatments. 


Figure 8.9: Images for Code Two 


360 


Chapter 8: Lab Example 2 


Overview 


In this lab example, we will utilize the Image System 


Engineering Toolbox for Biology, or ISETBio for short. 
ISETBio is a set of tools for MATLAB that can conduct visual system analysis 
by providing capabilities to create spectral radiance scenes and model their 
effects on human optics, eye movements, cone absorptions, retinal cell 
properties, and more. The link to download and install ISETBio from their 
GitHub can be found here: 


GitHub - isetbio/isetbio: Tools for modeling image systems engineering in the 


human visual system front end. 


You can learn more about ISETBio from Cottaris et al. [195]. 

As part of this lab, we will reinforce our understanding of the visual system 
anatomy by creating a plot of cone density in the eye. This will be 
accomplished by utilizing a tutorial script from the ISETBio toolbox in 
MATLAB. 


Requirements 


e MATLAB 
e ISETCam 
e ISETBio 


Description 


As we learned in the visual system chapter, cones are concentrated in the fovea. 
The cones are responsible for our color vision but are less light-sensitive than 
rods. For this reason, our anatomical knowledge is cleverly applied during 
times of darkness. This concentration of cones in the center of our vision 
creates a centrally focused blind spot at night. Instead of looking directly at an 
object when it is dark, you can look 5-10 degrees away from the center of gaze 
and have better light sensitivity, effectively enhancing your night vision. Let's 
look at a plot of the cone density to illustrate this concept effectively. 
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Within MATLAB, the path to the ISETBio tutorial/cones folder. We can find 
a “t_conesDensity.m” script file in this folder. If you open this, the ISETBio 
team has succinctly described the script's function and the data's source. They 
use two resources to obtain the cone density estimates Curcio et al. [170] and 
Song et al.[171]. 

The script file is short and to the point. A quick evaluation of the script will 
show you how the ISETBio toolbox works to obtain the final goal of plotting 
cone density. The “pos” portion of the code specifies the location of some 
object outward from the fovea in millimeters. The “ang” portion defines the 
angle of view around the eye in radians. This allows us to plot a full range of 
the eye map (Fig. 8.10). Then, the distribution of cones from the two research 
papers above is defined. 

Run the script for t_conesDensity.m and evaluate the final logarithmic scale 
plot that is produced: 


Summary 


We utilized the ISETBio toolbox in MATLAB to run a basic tutorial that 
allowed us to visualize the concentration of cones across the retina. This 
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Figure 8.10: Concentration of Cones across the Retina 
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provides us with a basic understanding of how the ISETBio toolbox functions 
and may guide you in thinking of creative ways to employ this toolbox for 
your own research goals. 
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The Eyes Have it: Unlocking 
Secrets with Eye Trackers 


364 


Introduction and Learning Objectives 


Eye tracking is a unique and essential tool with many uses in neuroengineering 
applications. Eye tracking generically refers to measuring gaze, where 
someone is looking, or the motion of an eye. It enables a researcher to 
investigate various visual and cognitive functions. Additionally, it pairs well 
with other neuroengineering techniques to provide robust and detailed data. 
The basic principle of eye tracking involves utilizing hardware to detect and 
record eye movements. This approach can allow researchers such as yourself 
to analyze how the eyes interact with visual stimuli. Eye tracking objectively 
measures where and how long an individual looks at specific locations or 
objects. This can be used to guide engineering designs or assist in 
neuropathology diagnosis. 

This chapter will explore various applications of eye tracking in engineering. 
By the end of this chapter, you’1l be able to understand the following concepts: 


Understand the anatomical and engineering principles of eye- 
tracking technology. 


Identify basic hardware components of eye trackers. 
Describe various applications of eye-tracking experimentation 
across multiple fields. 


Review of Anatomy Applied to Eye Tracking 


Building on our previous chapter's in-depth review of visual system anatomy, 
let’s explore some important eye-tracking specifics. The primary structures in 
eye tracking include the cornea, lens, retina, iris, pupil, and optic nerve. 


Cornea 


The transparent front layer collects incoming light and directs it to the retina. 


Lens 


It is located behind the cornea and fine-tunes the focus of light onto the retina. 
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Retina 

A layer of photoreceptor cells at the back of the eye detects light and converts 
it into electrical signals sent to the brain. 

Pupil 

The dark opening of the eye that is formed by the iris. 

Tris 

The colored part of the eye controls the size of the pupil and the amount of 
light entering the eye. 

Optic Nerve 


Transmits visual information from the retina to the brain for processing. 
Understanding the basic anatomical structures and their functions helps 
understand how eye tracking works, designing experiments, and interpreting 
results. For instance, changes in pupil size can indicate variations in emotional 
state, while abnormalities in eye movement patterns can signal neurological 
issues. 


Eye Tracking Technology 
Basics of Eye Movement 


You should know a few unique eye movements that can be important when 


understanding eye tracking, formulating experiments, or interpreting eye- 
tracking data. 


Saccades 


These are rapid eye movements that orient the gaze toward something. These 
movements are notable for tasks such as reading, where the eyes jump from 
word to word. 

Nystagmus 

The involuntary, rhythmic movement of the eyes, often in response to 
neurological conditions. Nystagmus can occur in the horizontal, vertical, or 
rotational plane. This movement type is commonly associated with impairment 
and can indicate vestibular dysfunction. 
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Fixations 


Fixation occurs when the eyes are stationary, allowing for detailed visual 
processing. 


Smooth Pursuit 


These are slower, controlled eye movements that track moving objects. 
Smooth pursuits are crucial for sports and driving, where continuous tracking 
of moving objects is necessary. Lack of smooth pursuit is another common 
vestibular dysfunction indicator. 

Understanding these different eye movements is vital for eye-tracking research 
as they provide insights into the underlying neural mechanisms of visual 
processing. 


Types of Eye Trackers 


Eye trackers can be classified into three categories based on their design and 
application: remote, head-mounted, and integrated systems. Examples of these 
devices are shown in Fig 9.1 [198]. Each type has distinct advantages and 
limitations that make them suitable for specific research and practical 
applications. 

Remote eye trackers are portable, consumer-level devices that track eye 
movements from a distance. They are typically mounted on a table or 


Figure 9.1: Types of eye trackers [198] 
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integrated into laptops or webcams, making them convenient for various 
settings in controlled environments or comfortable living spaces. However, 
their accuracy can be lower than that of more specialized systems due to 
limitations in camera quality or excessive motion. 

Table-mounted eye trackers are differentiated from “remote-eye trackers” by 
being research-grade hardware. These trackers are also placed on a desk or 
table to face the participant. They use high-resolution cameras and commonly 
infrared light to capture detailed images of the eyes. Table-mounted eye 
trackers offer higher accuracy and precision than remote systems and are more 
often used in controlled laboratory environments for more controlled research. 
Head-mounted eye trackers are worn on the participant’s head as standalone 
devices or integrated into glasses, helmets, or other headsets. These systems 
offer greater mobility and the ability to track eye movements in real-world 
settings with higher quality and accuracy than remote systems. Glasses-based 
trackers are lightweight devices with small cameras that track the wearer’s eye 
movements, suitable for studies involving driving, sports, and more. 
Integrated eye trackers are becoming more widespread in virtual reality (VR) 


Figure 9.2: Integrated eye trackers for VR systems [199] 


or augmented reality (AR) headsets like Fig. 9.2 [199]. VR/AR headsets with 
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eye tracking open new research avenues, including studying responses to 
simulated environments or placing subjects into otherwise difficult or unsafe 
real-life settings. The potential for studying human responses in various 
atmospheres presents unique implications for future research. 

Eye tracking is also increasingly integrated into automotive systems and 
consumer electronics, such as cars and smartphones. Each type of eye tracker 
has its advantages and disadvantages, which will be discussed later. 


Hardware Components 


Modern eye trackers are sophisticated devices that rely on advanced hardware 
components to measure and analyze eye movements with high precision. 
These components include high-resolution cameras, infrared light sources, and 
various supporting technologies that capture detailed and accurate data on 
where and how users look. 


High-Resolution Cameras 

The core of any eye-tracking system is its high-resolution camera, which cap- 
tures detailed eye images, particularly focusing on the pupil and corneal re- 
flections. The resolution of these cameras is critical as it determines the clarity 
and precision with which the eye's movements are recorded. Modern eye track- 
ers typically operate at high frame rates, often around 1000Hz (1000 frames 
per second). This high frame rate is essential for accurately tracking rapid eye 
movements, such as saccades, which are quick, simultaneous movements of 
both eyes in the same direction. Capturing these movements requires a fast 
camera and the ability to process a large amount of data in real-time. 
High-resolution cameras ensure that even the smallest eye movements, such 
as micro-saccades, are detected. This spatial accuracy is vital for applications 
requiring precise eye movement data, such as psychology, neuroscience, and 
human-computer interaction research. 


Infrared Light Sources 


Infrared light sources are used to illuminate the eye, specifically the pupil, to 
enhance contrast and improve the accuracy of eye-tracking measurements. In- 
frared light creates a distinct reflection from the cornea, known as the corneal 
reflection or "glint," which is used with pupil detection to determine the gaze 
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direction. One significant advantage of infrared light is that it is invisible to 
the human eye, meaning it does not interfere with the user’s natural vision or 
tasks. This allows the eye tracker to operate without distracting the user or 
altering their behavior, which is crucial for maintaining the integrity of the data 
collected. Infrared illumination reduces the impact of ambient lighting condi- 
tions, which can vary significantly between different environments. Whether 
the user is in a brightly lit room or a dimly lit space, the infrared light ensures 
consistent eye-tracking performance by providing uniform eye illumination. 
This consistency is particularly important in applications like usability testing 
or driver monitoring, where lighting conditions may be unpredictable. Eye 
trackers with infrared light are significantly more accurate than those without. 
The enhanced contrast provided by infrared illumination allows for more pre- 
cise detection of the pupil and corneal reflections, leading to better tracking of 
gaze direction and eye movements. This increased accuracy is critical in ap- 
plications where even minor errors can lead to significant misinterpretations, 
such as in cognitive research or medical diagnostics. Without infrared illumi- 
nation, eye trackers are more susceptible to errors caused by changes in ambi- 
ent light, reflections, or shadows on the face. These factors can distort the eye's 
image and lead to inaccurate measurements. Infrared light mitigates these is- 
sues by providing a consistent lighting source, reducing the likelihood of track- 
ing errors, and improving the overall reliability of the data. Infrared-enhanced 
eye trackers offer better usability in a wider range of conditions. They are less 
dependent on the external environment, making them suitable for various set- 
tings, from controlled laboratory environments to more dynamic real-world 
scenarios. This flexibility is essential for expanding eye-tracking technology 
in virtual reality, where lighting conditions vary dramatically. 

Integrating high-resolution cameras and infrared light sources in modern eye- 
trackers represents a significant advancement in eye-tracking technology. 
These components work together to provide accurate, reliable, and consistent 
tracking of eye movements, even in challenging conditions. The benefits of 
infrared-enhanced eye trackers, including increased accuracy, reduced errors, 
and enhanced usability, make them indispensable tools across various applica- 
tions, from cognitive research and human-computer interaction to virtual 
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reality and medical diagnostics. As eye-tracking technology continues to 
evolve, the role of these core hardware components will remain critical in en- 
abling new and innovative uses of eye-tracking data. 


Software for Eye Tracking 


Eye-tracking software is responsible for collecting and processing data from 
the hardware. This is where the pupillary detection and gaze direction are truly 
handled. The software uses sophisticated algorithms to calculate the gaze 
direction and record the data for further analysis. The software can also 
manage artifacts such as unintentional motion or detect blinking. 

Modern eye-tracking systems offer real-time data acquisition and can provide 
immediate feedback on eye movements. This real-time capability is essential 
for applications such as user interface design, where immediate adjustments 
based on eye movement data can enhance usability and performance. 


Algorithms and Mathematical Models 


Various algorithms and math models are applied to detect and track the pupil, 
gaze correctly, or blink. Different algorithms can be used to accomplish 
different goals. Some common methods include circular Hough transform, I- 
VT, stereo geometry, convolution neural network, or other machine learning 
approaches. 

Pupil Detection 

Many algorithms have been developed to detect the pupil itself. A commonly 
discussed algorithm for this application is the circular Hough transform, 
although there are many more [200], [201]. The detection of the pupil and 
measurement of pupil velocity can be used to distinguish fixations, saccades, 
smooth pursuit, or other eye movements we previously discussed[202]. 

Gaze Mapping 

Gaze mapping is a very common application of eye-tracking models. This 
involves translating eye movement data into meaningful information, like 
determining where a subject is looking on a screen and for how long [203], 
[204]. This process requires calibration, where the eye tracker is adjusted to 
each individual’s eyes to ensure accurate mapping of gaze points. This can 
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produce “heat maps” showing concentrations of fixation and patterns of focus 
to determine how subjects typically interpret something. This can have a 
variety of applications, such as designing a research poster, a movie poster, an 
advertisement, or many other applications. There are many approaches to 
utilizing or analyzing eye-tracker data. Gaze mapping is just one common and 
easily applicable approach. 

Corneal Reflection 

An interesting aspect of gaze mapping is called corneal reflection. This 
involves utilizing the reflection of the infrared light source on the cornea, 
called a glint, to determine gaze direction. This approach analyzed the position 
of the corneal reflection to the pupil’s center to calculate an accurate gaze 
direction. 

This is not an all-encompassing overview, and it is important to appreciate new 
advancements occurring with the application of deep-learning artificial 
intelligence technologies to perform even more accurate eye-tracking analysis. 
These AI techniques can harness large datasets and information from previous 
models to perform even more precise and robust analyses within eye-tracking 
systems. 


Experimental Design 

Setting up an eye-tracking experiment involves several steps that may vary 
depending on the technology you are employing and the environment within 
which you are using it. 

Calibration 

Each participant must undergo a calibration process where they are asked to 
look at known points on a screen. This process adjusts the eye tracker for 
individual eye anatomy and movement pattern differences. 

Lighting Conditions 

Optimal lighting is crucial for accurate eye tracking, especially when using 
lower-quality hardware. While infrared illumination helps mitigate the impact 
of ambient light, the experimental setup should avoid direct light sources that 
can create reflections and interfere with tracking. In dark environments, using 
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green or red light can provide illumination without interfering with night- 
vision sensitivity or other responses. 

Participant Instructions 

Subjects should be given clear instructions to ensure they understand the tasks 
and the importance of maintaining a stable head position during tracking. 


Performance and Limitations 


Accuracy and Precision 


Accuracy within the realm of eye trackers refers to the degree to which the 
measured gaze corresponds to the actual point of fixation. Precision, on the 
other hand, refers to the consistency and repeatability of the measurements. 
High accuracy and precision are critical for detailed analysis of eye 
movements. Factors affecting accuracy and precision have already been 
touched on, including the hardware's quality, the software's effectiveness, and 
the experimental setup. 

Pupil Size and Eye Shape 

Variations in pupil size and eye shape can impact the accuracy of eye tracking. 
For example, larger pupils can be easier for the eye tracker to detect, improving 
accuracy. However, changes in pupil size due to lighting conditions or 
cognitive load can introduce variability. Additionally, iris color can play a role 
in pupil size and response. Individuals with lighter-colored eyes may have 
more constricted pupil size in certain lighting conditions than dark eyes in the 
same environment. The degree of iris constriction or dilation can also vary 
from person to person and affect measurements. 


Glasses and Contacts 


Eyewear can create reflections and distortions, making eye tracking difficult 
depending on the hardware selection. Advanced algorithms may compensate 
for this without issue, and hardware selection, such as wearable eye-trackers, 
may negate this issue if coupled with prescription lenses. 
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Applications in Research 

Cognition 

Eye tracking can provide insights into cognitive processes by revealing how 
attention is allocated and prioritized. For example, eye movements can indicate 
how individuals process information and how long they process it while 
reading, solving problems, or making decisions. This can be paired with other 


experimental techniques, such as the fMRI, to provide several layers of 
analysis that can be interpreted in tandem. 


Human Factors 


The human factor is a meaningful topic in many different fields. In the 
automotive field, for example, human factors research may be concerned with 
innumerable safety questions: 

1. How and when people respond to signage while driving. 

2. Driver behavior during emergencies. 

3. Driver behavior in novel roadway designs. 

4. How intoxication alters driver response times. 
This is just a short list outlining the potential applications of eye-tracking 
research in ways that can benefit society. Additional fields include aviation, 
gaming, and even the film industry. 


Psychology 


In psychology, eye tracking can be used to study emotion, social, or 
developmental processes. For example, researchers can investigate how 
autistic individuals process social cues by analyzing their eye movements 
when viewing faces. Eye tracking can also be used to study attention biases in 
anxiety and depression by examining how individuals respond to emotional 
stimuli. 


Available Products on the Market 


Several common eye-tracking products have unique features and capabilities 
that range from consumer to research-grade. When selecting an eye-tracking 
system, it is important to consider the advantages and disadvantages of each 
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product to determine the best fit for your research or application. With that 
said, the first step is identifying your experimental design and, thus, your re- 
search environment. The major criterion in this regard is the level of portabil- 
ity, as this is the major limitation of the device you select. The major products 
available today are discussed, but the list is not exhaustive. 

Tobii 

Tobii is a well-known company that produces eye-tracking products and is best 
known for its glasses-based system [8]. Their glasses contain several cameras 
and infrared light sources and utilize corneal reflection, pupil, and stereo 
geometry to perform accurate eye tracking and analysis. They also have 
screen-based eye trackers that can be used in various applications, making 
them a versatile option for consumer and research purposes. 

The Tobii products are user-friendly and are offered at a range of applications 
from consumer-level to research-grade. The glasses-based systems are partic- 
ularly valuable for naturalistic studies, as they allow for real-world eye track- 
ing with minimal intrusiveness. Naturally, with this level of portability, battery 
life becomes a potential disadvantage. Additionally, it is specified that data 
should be sampled up to 100 Hz, which could be a disadvantage depending on 
the sampling level your research requires. 


SR-Research 


Another well-researched eye-tracking product is the EyeLink system by SR 
Research. The EyeLink products are head-mounted or table-mounted, as 
shown in Fig. 9.3 [205]. These devices provide high resolution and accuracy 
and utilize pupil detection and corneal reflection methods, making them 
suitable for highly detailed studies, such as those in cognitive neuroscience or 
psychophysics. 

The SR-Research EyeLink is well known for its high accuracy and resolution. 
As such, their system is reported to have a 2000 Hz sampling rate and 0.15- 
degree accuracy. The disadvantage comes both at cost and complexity levels. 
Additionally, it may not be suitable for naturalistic studies like the Tobii. 
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iMotions 

iMotions has several eye trackers, including screen-based, glass, VR, and 
webcam-based systems [206]. Their systems harness various algorithms and 
models, allowing for eye-tracking applications that best suit your experimental 
design. This flexibility makes iMotions a strong choice for researchers 
wanting customizable solutions for multiple contexts. The flexibility of their 
eye-tracking offerings is a clear advantage for researchers. The cost of these 
systems may provide limitations for certain budgets. 


Pupil Labs 


Pupil Labs is another company with various eye tracker systems similar to 
iMotions [207]. Their eye-tracking systems harness diverse algorithms and 
configurations that support various research needs. Pupil Labs also places a 
strong emphasis on open-source solutions, which is called Pupil Core. This can 
be extremely advantageous for researchers who value open-source alternatives 
and cost-effective solutions. This enables high versatility that can be adapted 
to various experimental designs. Although some consider the open-source 
nature an advantage, others consider this a disadvantage, and it is up to each 


Figure 9.3: SR-Research table mounted eye tracker from SR-Research [205] 
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user to make that decision. The primary limitation of this could be the support 
and resources provided. 

These different products can help you accomplish unique research goals in 
various environments. This allows a researcher to develop novel designs and 
applications that can help answer specific research questions. 


Emerging Trends in Eye Tracking Tech 


A discussion on emerging trends for any technology in 2024 is incomplete 
without including artificial intelligence (AI). Recent advancements in the AI 
space, along with machine learning, have begun to enhance current eye-track- 
ing technology. A Google search of the topic will further solidify this state- 
ment. First, AI can be harnessed to improve the speed and precision of eye- 
tracking technology. This can be in the form of improved detection algorithms 
or improved sensitivity to eye movements. 

Additionally, AI could be used to detect and classify eye movements and pre- 
dict eye movements. This idea is already being implemented in “Neuromar- 
keting” methodologies. This idea will be further discussed in the section on 
ethical considerations. 

In the medical space, Al-enhanced technology can improve medical diagnos- 
tics through the previously discussed techniques. Due to the increased sensi- 
tivity and accuracy, early-stage detection of neurological disorders may be- 
come a reality. Given the fairly low-cost implementation of eye trackers rela- 
tive to other medical technology, this can provide broader consumer-level 
treatment. 

Although this chapter briefly touched on VR and AR, it is still a “new” and 
growing field. The growing incorporation of eye tracking in these systems will 
surely open new avenues for research in neuroengineering. As VR/AR and AI 
technology further integrate, one can foresee incredibly realistic immersive 
environments for neurological simulation studies that will produce more real- 
istic results. 

Another trend in eye technology, and by extension eye-tracking, is its applica- 
tion to biometrics. Currently, cellphones utilize “face” or fingerprint 
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recognition. The use of eye biometrics is a foreseeable application. Some po- 
lice departments are already using eye biometrics for identification purposes. 


Challenges and Future Directions 


Accuracy and Precision of Eye Trackers 


The level of accuracy and precision is always a considerable factor when a 
researcher implements certain technologies. Eye-trackers' goals within the re- 
search space will always be high accuracy and precision. The previous discus- 
sion on AI and machine learning in space opens an avenue for achieving this 
goal. For those interested in implementing eye-trackers in neuroengineering, 
the tremendous growth in this technology will provide several opportunities 
for novel research ideas. 

Additionally, using IR may not fully address the challenge of collecting eye- 
tracking data in low-light conditions. Additional approaches may be developed 
to address some of the challenges faced during low-light conditions. 


Ethical Considerations and Privacy 


There are ethical considerations and data privacy concerns with integrating 
eye-tracking technology into everyday technology. Personalized iris data, eye 
movement data, and, by extension, medical can reveal sensitive information 
about an individual. There will be a need for clear guidelines and regulations 
that follow current human subject testing guidelines on collecting and storing 
personal data. This also means researchers applying this technology must pri- 
oritize transparency and consent to prevent misuse of this information. Addi- 
tionally, with eye-tracking technology being potentially used for biometrics, 
the public should be confident in the accuracy and privacy considerations that 
are being considered. 


Multimodal Integration in Neuroengineering Research 


As we have covered in this chapter, eye tracking is uniquely powerful in its 
portability and affordability. The multimodal integration of eye-tracking with 
other methods, such as fMRI, fNIRS, or EEG, could provide a more 
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comprehensive understanding of cognitive and emotional processes. This 
again presents unique research ideas and challenges for the future. 


Chapter 9: Summary 


Eye tracking technology is unique, affordable, accessible, and has applications 
in any scientific field. The portability and application in naturalistic 
environments provide the option to develop any number of unique studies that 
have the potential to benefit society. Technological advancements, especially 
with VR headsets and AI, promise even more applications in future research. 
Whether your interests lie in psychology, medicine, or engineering, there is 
room for eye-tracking research in each field. 

In the next journey of this book, you will explore the worlds of psychophysics 
and virtual reality. This does not mark the end of your discussion of eye- 
trackers. It highlights just how applicable eye-tracking technology is across 
various domains. For those of you with a special interest in eye-tracking 
research, the multimodal integration possibilities within the psychophysics 
space will undoubtedly pique your interest. 
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Chapter 9: Learning activities 


Learning Activity 9.1 


Learning Activity: Drawing a Functional 
Block Diagram (FBD) of an Eyetracker 


Objective: 


Students will understand the components and functionality of an eye-tracker 
by creating a functional block diagram (FBD) and discussing their designs 
with peers. 


Materials Needed: 


Paper 

Pens/Pencils 

Markers (optional) 

Reference materials on eye trackers (e.g., textbooks, articles, 
diagrams) 


Activity Outline: 


1. Introduction (10 minutes) 

o Briefly introduce the concept of eye trackers and their 
applications in various fields, such as psychology, 
marketing, and UX/UI design. 

o Explain what a Functional Block Diagram (FBD) is and its 
importance in illustrating the components and workings of a 
system. 

2. Individual Work (20 minutes) 

o Each student will individually draw an FBD of an eye- 
tracker. They should include and label the main components, 
such as the camera, infrared light source, processing unit, 
and software interface. They should also show the data flow 
from the eye movement to the final output on the computer 
screen. 
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(2) 


Encourage students to consider the interactions between 
different parts and the system’s workflow. 


3. Pair Share (15 minutes) 


(2) 


Students will pair up with a classmate to share and discuss 
their FBDs. Each pair should compare their diagrams, 
discuss the differences and similarities, and provide 
constructive feedback to each other. 

Pairs should refine their diagrams based on the discussion, 
ensuring all essential components and data flows are 
included. 


4. Class Discussion and Reporting (15 minutes) 


(2) 


Each pair will present their refined FBDs to the class. They 
should explain the components they included and the 
rationale behind their design. 

After each presentation, allow time for questions and 
comments from the class. This will help identify any 
common misconceptions and clarify any doubts. 


5. Conclusion (5 minutes) 


(e) 


Assessment: 


Summarize the key points discussed during the class 
presentations. 

Highlight any particularly well-done diagrams and explain 
why they stood out. 

Encourage students to continue exploring how different 
systems are designed and function by creating FBDs for 
other devices. 


e Participation: Ensure that each student actively participates in the 
individual work, pair share, and class discussion. 

e Accuracy and Completeness: Evaluate each student’s FBD for 
accuracy and completeness, ensuring all main components are 
included and correctly labeled. 

e Collaboration: Assess how well students work in pairs and 
contribute to the class discussion. Learning activity 9.2 


Learning Activity 9.2 
Evaluating an Eyetracker Game 


Objective 


Students will gain hands-on experience with an eye-tracker 
through an interactive game and evaluate its effectiveness and functionality by 
working collaboratively. 


Materials Needed 


e Computers or tablets with internet access 
e Access to the eye-tracker game at GazeRecorder 
e Paper and pens/pencils for note-taking 


Activity Outline 


1. Introduction (10 minutes) 

e Introduce the concept of eye trackers and their applications in var- 
ious fields. 

e Explain the purpose of the activity: to explore and evaluate an eye- 
tracker game. 

2. Individual Game Play (20 minutes) 

e Each student will individually access the eye-tracker game on 
GazeRecorder. 

e They will spend time playing the game, paying close attention to 
how the eye-tracker responds to their eye movements, the accu- 
racy, and the user interface. 

e Students should take notes on their experiences, focusing on the 
game's functionality, usability, and any issues encountered. 

3. Pair Share (15 minutes) 

e Students will pair up with a classmate to discuss their experiences 
with the game. 

e Inpairs, students should compare notes and discuss the following: 

e How accurately did the eye-tracker follow their eye 
movements? 


382 


e Any challenges or difficulties they faced while playing 
the game. 

e Suggestions for improvements or features they found par- 
ticularly useful. 


4. Class Discussion and Reporting (15 minutes) 


Each pair will present their findings to the class. 

Presentations should cover the main points of their discussion, 
highlighting both positive aspects and areas for improvement. 
Encourage other students to ask questions and add their observa- 
tions. 


5. Conclusion (5 minutes) 


Summarize the key points discussed during the class presenta- 
tions. 

Highlight common themes or unique insights from the evalua- 
tions. 

Discuss the potential applications of eye-tracker technology in 
various fields and how such games can be used for research or 
training. 


Assessment 


Participation: Ensure that each student actively participates in the in- 


dividual gameplay, pair share, and class discussion. 


Quality of Evaluation: Evaluate each pair's presentation for thor- 


oughness, clarity, and insightfulness. 


Collaboration: Assess how well students work in pairs and contribute 


to the class discussion. Learning activity 9.3 
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Learning Activity 9.3 


Creating and Analyzing Heat Maps with an 
Eyetracker 


Objective 


Students will learn how to create heat maps using an eye-tracker app based on 
their webcams, understand the calibration process, and discuss their experi- 
ences and findings with the class. 


Materials Needed 


e Computers with internet access and webcams 

e Access to an eye-tracker app that can generate heat maps (e.g., Gaz- 
eRecorder, WebGazer.js) 

e Paper and pens/pencils for note-taking 


Activity Outline 


1. Introduction (10 minutes) 

e Briefly introduce eye-tracking technology and its applications, fo- 
cusing on how heat maps can provide insights into user behavior 
and preferences. 

e Explain the purpose of the activity: to create a heat map of their 
favorite webpage and understand the calibration process. 

2. Individual Activity (30 minutes) 

e Selecting a Webpage: 

o Each student will choose their favorite webpage to ana- 
lyze (e.g., a news site, social media page, or any fre- 
quently visited website). 

e Calibration: 

o Before creating the heat map, students need to calibrate 
the eye tracker app. Explain the importance of calibration: 
it ensures the eye-tracker can accurately detect where the 
user is looking on the screen. 
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[e) 


Students should follow the app’s instructions to complete 
the calibration sequence, which typically involves look- 
ing at specific points on the screen. 


e Creating the Heat Map: 


(2) 


After calibration, students will navigate their chosen 
webpage while the eye-tracker records their eye move- 
ments. 

The app will generate a heat map based on where they 
frequently look on the page. 


e Note-Taking: 


= Students should take notes on the following: 

« Their experience with the calibration 
process: Was it easy or difficult? Why is 
calibration necessary? 

= Observations about the generated heat 
map: Which areas of the webpage 
attracted the most attention? Were there 
any surprises? 

= Thoughts on the overall experience and 
potential improvements. 


e Class Discussion and Reporting (20 minutes) 


(e) 


(e) 


Each student will present their findings to the class, 
addressing the questions about the calibration sequence 
and their heat map observations. 

Encourage students to compare their heat maps and 
discuss common patterns or differences. 

Facilitate a discussion on the importance of calibration 
and how it affects the accuracy of the eye-tracker. 


e Conclusion (5 minutes) 


(2) 


Summarize key points from the class discussion, 
highlighting common insights and unique observations. 
Discuss the broader implications of eye tracking 
technology and heat maps in understanding user 
behavior and improving webpage design. 

@ 


rr) 
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Chapter 9: Lab introduction 


In this series of lab exercises, you will explore eye-tracking and the 
simulation of eye movements using MATLAB. These labs are 
designed to provide hands-on experience with image processing and 
dynamic system modeling, deepening your understanding of visual 
tracking and eye movement dynamics. 

You will run a simplistic eye-tracking exercise in the first lab using 
MATLAB. The focus will be on implementing the Hough transform 
and Viola-Jones algorithms. The Viola-Jones algorithm detects and 
classifies faces, noses, eyes, mouths, and the upper body in images, 
giving you practical experience in facial feature detection and 
classification. 

In the second lab, you will create MATLAB code to generate 
simulated saccades, which are rapid, ballistic eye movements as they 
shift focus from one point to another. Using a second-order system, 
you will develop a function that models saccades’ acceleration and 
deceleration phases. Additionally, you will demonstrate code for 
detecting saccades, providing a comprehensive understanding of 


both simulating and identifying these rapid eye movements. 
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Chapter 9: Lab Example 1 


Overview 


In this lab, we will try to run a very simplistic eye-tracking 


exercise using MATLAB. The code we will use implements 
the Hough transform algorithm previously touched on, as 
well as an algorithm called Viola-Jones. MATLAB can use this algorithm to 
detect and classify faces, noses, eyes, mouths, and upper bodies. The base code 
is inspired and modified from this Github poster: 
https://github.com/ishitadatta/EyeballDetection 


Objective 

This laboratory exercise aims to understand how to apply eye-tracking 
algorithms within MATLAB to perform a simplistic eye-tracking exercise 
using a built-in webcam. 


MATLAB Setup 


Let’s discuss a basic MATLAB code to accomplish our laboratory exercise 
goals. Begin by clearing your workspace and any figures with the following: 


clear all 
clf('reset'); 


As we discussed in the objectives, we are going to use the built-in webcam 
from an LG Gram laptop, and we can call the webcam using: 


cam=webcam(); 
Next, we will set up some simple identifier’s that tell us when the subject is 
looking right, left, straight, or not found (noface). 


right=imread('RIGHT.jpg'); 
left=imread('LEFT.jpg'); 
noface=imread('no_face.jpg'); 
straight=imread('STRAIGHT.jpg'); 
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To detect the face and eyes, we are going to use the Viola-Jones algorithm 
with: 


detector = vision.CascadeObjectDetector(); 
detector1 = 
vision.CascadeObjectDetector('EyePairSmall'); 


Next, we define the laptop screen dimensions and camera information 
manually with the following definitions: 


screenWidthCm = 50; 
screenHeightCm = 30; 
distanceToCameraCm = 60; 
cameraFoV = 60; 


We need to create an array to store the coordinates of the eye position with 
respect to time, and this data will be collected for 30 seconds. The repeat 
variable, while true, will let us run a continuous loop of the eye tracking: 


pupilPositions = []; 
startTime = tic; 
duration = 30; 
repeat = true; 


Let us now define the loop that will run for the 30-second duration; this portion 
defines the capture of images, face and eye detection, and pupil detection via 
the Hough transform algorithm, defines bounding boxes gaze detection, and 
allows the loop to end: 


while repeat 
vid=snapshot (cam) ; 
vid = rgb2gray(vid); 
img = flip(vid, 2); 


bbox = step(detector, img); 


388 


if ~ isempty(bbox) 
biggest_box=1; 
for i=1:rank(bbox) 
if bbox(i,3)>bbox(biggest_box, 3) 
biggest_box=i; 
end 
end 
faceImage = imcrop(img, bbox(biggest_box,:)); 
bboxeyes = step(detector1, faceImage) ; 


subplot(2,2,1),subimage(img); hold on; 
for i=1:size(bbox,1) 
rectangle('position', bbox(i, :), 
"lineWidth', 2, ‘edgeColor', ‘y'); 
end 


subplot (2,2,3),subimage(facelImage) ; 
if ~ isempty(bboxeyes) 


biggest _box_eyes=1; 
for i=1:rank(bboxeyes) 
nap 
bboxeyes(i, 3) >bboxeyes(biggest_box_eyes, 3) 
biggest _box_eyes=i; 
end 
end 


bboxeyeshalf=[bboxeyes(biggest_box_eyes,1), bboxeyes(bigg 
est_box_eyes,2),bboxeyes(biggest_box_eyes,3)/3,bboxeyes( 
biggest_box_eyes,4)]; 


eyesImage = imcrop(facelmage, bboxeyeshalf(1,:)); 
eyesImage = imadjust(eyesImage) ; 


r = bboxeyeshalf(1,4)/4; 
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[centers, radii, metric] = imfindcircles(eyesImage, 
[floor(r-r/4) floor(r+r/2)], ‘ObjectPolarity','dark', 
"Sensitivity', @.93); 

[M,I] = sort(radii, 'descend'); 


eyesPositions = centers; 


subplot(2,2,2),subimage(eyesImage); hold 
on; 


viscircles(centers, radii, 'EdgeColor','b'); 


if ~isempty(centers) 
pupil x=centers(1,1); 
pupil y=centers(1,2); 
pupilPositions = [pupilPositions; 
pupil_x,pupil_y]; 

disL=abs(@-pupil x); 
disR=abs(bboxeyes(1,3)/3-pupil_ x); 
subplot(2,2,4); 
if disL>disR+16 

subimage(right) ; 
else if disR>disL 

subimage(left) ; 

else 

subimage(straight) ; 

end 

end 


end 
end 
else 
subplot(2,2,4); 
subimage (noface) ; 
end 
set(gca, 'XtickLabel',[],'YtickLabel',[]); 
hold off; 
if toc(startTime) > duration 
repeat = false; 
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end 
end 


Now that we have defined most of our eye-tracking MATLAB process and 
data collection, we will normalize the pupil position to the screen dimensions 
that we previously defined. This enables us to map the gaze position on the 
screen (roughly). This can be accomplished with the following: 


if ~isempty(pupilPositions) 
imgWidth = size(vid, 2); 
imgHeight = size(vid, 1); 


normalizedPupilX = pupilPositions(:,1) / imgWidth; 
normalizedPupilY = pupilPositions(:,2) / imgHeight; 


screenPosX = normalizedPupilX * screenWidthCm; 
screenPosyY = (1 - normalizedPupily) * 
screenHeightCm; 
end 


We can utilize a Gaussian kernel to smooth the data that was collected: 


kernelSize = 25; 
Sigma = kernelSize / 5; 


[x, y] = meshgrid(linspace(-kernelSize/2, kernelSize/2, 
kernelSize), linspace(-kernelSize/2, kernelSize/2, 
kernelSize) ); 

gaussianKernel = exp(-(x.42 + y.%2) / (2 * sigma’2)); 
gaussianKernel = gaussianKernel / sum(gaussianKernel, 
fall) 


This ends the eye-tracking MATLAB code segment. The commands above 
will detect the face, eyes, and pupils and collect an array of coordinates for 
their position on the screen. Let’s test the command in MATLAB; it should 
generate a window that detects whether you look left, right, or straight! It 
should look like Fig. 9.4. 
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Figure 9.4: Eye Tracking 


To analyze and process the data one step further, we could try to generate a 
heat map representing a visual estimation of gaze points on the laptop screen, 
as if you were looking at an image. This plot can be accomplished with: 


if ~isempty(screenPosxX) && ~isempty(screenPosyY) 
gridSizexX = 100; 
gridSizeY = 100; 


[N, Xedges, Yedges] = histcounts2(screenPosXx, 
screenPosY, [gridSizex gridSizeY]); 


density = N / sum(N, ‘all'); 
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smoothedDensity = conv2(density, gaussianKernel, 
"same'); 


[X, Y] = meshgrid(linspace(@, screenWidthCm, 
gridSizex), linspace(@, screenHeightCm, gridSizeyY)); 


figure; 
contourf(X, Y, smoothedDensity', 20); 
colormap( ‘hot’ ); 
colorbar; 
title('Smoothed 2D Heatmap of Gaze on Screen'); 
xlabel('Screen Width (cm)'); 
ylabel('Screen Height (cm)'); 
axis equal; 
else 
disp('No screen position data available for heatmap 
generation.'); 
end 
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Figure 9.5: Smoothed 2D Heatmap 


The results of running this 30-second code could produce a rough estimation 
of your gaze in a heat map of the screen dimensions. 
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To experiment with mapping my gaze while looking at an image, let’s take a 
full-screen zoom of this chapter’s title image and map my gaze while looking 
at the title image shown on the right: 

The result, if we were to plot the heat-map as an overlay to the original 
image (Fig 9.6) (bottom toolbar cropped out), shows us a rough idea of how 
my gaze was concentrated on the image: 


Figure 9.6: Heatmap on Image 
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Chapter 9: Lab Example 2 


Saccades are rapid, ballistic movements of the eyes that shift 
the point of fixation from one location to another. The human 
body produces These movements fastest, with peak 


velocities ranging from 30 to 500 degrees per second and 
durations typically between 20 to 200 milliseconds, depending on the 
amplitude. Saccades can be voluntary, such as when we look at a specific 
object, and involuntary, occurring reflexively in response to stimuli. Despite 
their speed, saccades are highly precise, allowing the eyes to land accurately 
on the intended target. The latency, or the time from a stimulus's appearance 
to the saccade's initiation, is usually around 200 milliseconds. 

In eye tracking, saccades are crucial because they help identify periods of 
fixation when the eyes are relatively still. Researchers and designers can gain 
insights into what captures and holds a person's attention by analyzing these 
fixations. The frequency, amplitude, and duration of saccades also indicate 
cognitive load and attentional processes; for instance, increased saccadic 
activity might suggest higher cognitive processing or searching behavior. In 
usability studies, understanding gaze patterns and how users shift their gaze 
can inform improvements in interface design and overall user experience. 
Saccades are integral to understanding visual perception in psychophysics, 


Figure 9.7: Saccade 


which explores the relationship between physical stimuli and the resulting 
sensations and perceptions. During saccades, vision is temporarily 
suppressed—a phenomenon known as saccadic suppression—enabling 
researchers to study how visual information is integrated across different 
fixations. Saccadic patterns also provide insights into attention and awareness, 
revealing how attention is shifted and maintained during various tasks. 
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Additionally, the latency and dynamics of saccades are used in reaction time 
studies to investigate the timing and coordination of perceptual and motor 
processes. 

Applications of saccadic research span various fields, including neuroscience, 
marketing, virtual reality, and gaming. In neuroscience, abnormal saccadic 
movements can indicate neurological conditions such as Parkinson's disease, 
schizophrenia, and ADHD. In marketing, eye tracking and analysis of saccadic 
movements help understand consumer behavior, such as how people view 
advertisements or product placements. In virtual reality and gaming, eye 
tracking enhances user experiences by allowing systems to respond to where 
users are looking in real-time. Understanding and analyzing saccades provide 
valuable insights into attention, perception, cognitive processes, and 
neurological health, making them a critical component of research and 
technological applications. 

A valuable resource for understanding the background of saccades is available 
at: 


SR Research Blog on Eye Tracking Terminology. 


In this lab, we will create MATLAB code to generate simulated saccades. 

To simulate a saccade in MATLAB, we need to create a function that models 
an eye's rapid, ballistic movement as it shifts focus from one point to another. 
Their quick acceleration and deceleration phases typically characterize 
saccades, and the dynamics can be approximated using a second-order system. 
We can modify the previous script to include multiple saccades with varying 
amplitudes and intervals to simulate a series of saccades. Here's an updated 
version of the script to achieve this: 


% Simulate a series of saccades in MATLAB 


% Parameters 

amplitudes = [10, -15, 20, -25, 30]; % amplitudes of the 
saccades in degrees 

durations = [@.05, 0.05, 0.05, 0.05, 0.05]; % durations of the 
Saccades in seconds 
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intervals = [@.1, 0.15, @.2, 0.1]; % intervals between 
saccades in seconds 
sampling rate = 1000; % samples per second 


% Total time calculation 
total_time = sum(durations) + sum(intervals); 
t_total = @:1/sampling rate:total_time; 


% Time vector and position initialization 
eye position = zeros(size(t_total)); 
current_time = @; 


% Second-order system parameters (assuming same for all 
saccades) 
zeta = 0.8; % damping ratio 


% Loop through each saccade 
for i = 1:length(amplitudes) 
% Current saccade parameters 
amplitude = amplitudes(i); 
duration = durations(i); 
omega = 2 * pi / duration; % natural frequency (rad/s) 


% Time vector for the current saccade 
t_saccade = @:1/sampling_rate:duration; 


% System transfer function: G(s) = omega*2 / (s‘%2 + 
2*zeta*omega*s + omega’*2) 

num = [omega’2]; 

den = [1, 2*zeta*omega, omega*2]; 

sys = tf(num, den); 


% Step response of the system to simulate the saccade 

[y, t_saccade] = step(sys, t_saccade); 

y = y * amplitude; % Scale the response to the desired 
amplitude 


% Update the eye position 

start_idx = find(t_total >= current_time, 1); 

end_idx = start_idx + length(y) - 1; 

eye_position(start_idx:end_idx) = 
eye_position(start_idx:end_idx) + y'; 
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% Update current time 
current_time = current_time + duration; 


% Add interval between saccades if not the last saccade 
if i < length(amplitudes) 
interval = intervals(i); 
current_time = current_time + interval; 
end 
end 


% Plot the series of saccades 

figure; 

plot(t_total, eye _position, ‘LineWidth', 2); 
xlabel('Time (s)'); 

ylabel('Eye Position (degrees)'); 
title('Series of Saccades Simulation’ ); 

grid on; 


% Optional: Save the plot as an image 
saveas(gcf, ‘series _of_saccades_simulation.png' ); 


% Display the plot 
disp('Series of saccades simulation completed.'); 


The script simulates a series of saccades with parameters that define the 
amplitudes (in degrees), durations (in seconds), and intervals between 
successive saccades (also in seconds). The simulation operates at a specified 
sampling rate (samples per second). To compute the total simulation time, the 
script sums the durations of all saccades and the intervals between them. It 
initializes a time vector and an eye position array to represent the total 
simulation period. 

The script processes each saccade within a loop by defining its natural 
frequency (omega) and damping ratio (zeta). It computes the step response of 
a second-order system for each saccade, updates the eye position array with 
the new saccade data, and adjusts the current time to account for the duration 
of the saccade and the interval to the next one. Finally, the script plots the eye 
position over time, visualizing the series of saccades. This allows for adjusting 
parameters to match the specific characteristics of the simulated saccades. 
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Now that we can generate saccades, it would be interesting to detect them as 
well. 

We can use a velocity-based method to detect saccades in the generated eye 
position data. This involves computing the velocity of the eye position and 
identifying points where the velocity exceeds a certain threshold, which 
indicates the occurrence of a saccade. 
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Figure 9.8: Series of Saccades Simulation 


% Saccade detection based on generated data from the previous 
script 


% Parameters for saccade detection 
velocity _threshold = 500; % degrees per second 
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% Compute the velocity of the eye position 

eye velocity = diff(eye_ position) * sampling rate; % velocity 
in degrees per second 

eye velocity = [@, eye velocity]; % pad the first element to 
match the length of eye_position 


% Detect saccades based on the velocity threshold 
saccade_indices = find(abs(eye velocity) > 
velocity_threshold) ; 


% Combine contiguous indices into saccades 

saccade starts = saccade_indices([1, 
find(diff(saccade_indices) > 1) + 1]); 

saccade _ends = saccade_indices([find(diff(saccade indices) > 
1), end]); 


% Plot the detected saccades 

figure; 

plot(t_total, eye position, ‘LineWidth', 2); 

hold on; 

for i = 1:length(saccade_starts) 
saccade _time = t_total(saccade_starts(i):saccade_ends(i)); 
saccade _pos = 

eye_position(saccade_starts(i):saccade_ends(i)); 
plot(saccade time, saccade_pos, ‘r', ‘LineWidth', 2); 

end 

xlabel('Time (s)'); 

ylabel('Eye Position (degrees)'); 

title('Detected Saccades'); 

legend('Eye Position’, ‘Detected Saccades'); 

grid on; 


% Optional: Save the plot as an image 
saveas(gcf, ‘detected_saccades.png'); 


% Display the results 
disp('Saccade detection completed.'); 
disp( ‘Detected saccades:'); 
for i = 1:length(saccade_starts) 

fprintf('Saccade %d: Start Time = %.3f s, End Time = %.3f 
s, Amplitude = %.2f degrees\n', ... 

i, t_total(saccade_starts(i)), 

t_total(saccade_ends(i)), 
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eye_position(saccade _ends(i)) - 
eye_position(saccade_starts(i))); 
end 


The script begins by calculating the velocity of the eye position using the diff 
function and scales it by the sampling rate to obtain the velocity in degrees per 
second. The eye velocity array is then padded with a zero at the start to 
match the length of the eye position array. 

For saccade detection, the script identifies indices where the absolute value of 
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Figure 9.9: Detected Saccades 


the velocity exceeds the velocity threshold. Contiguous indices are 
grouped into individual saccades by detecting where the difference between 
successive indices is greater than one. 
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The eye position is then plotted with the detected saccades highlighted in red, 
and each detected saccade's start and end times are displayed in the console. 
Finally, the script prints each detected saccade's start time, end time, and 
amplitude. This script should be run after the previous script that generates the 
saccade data. Adjust the velocity threshold as needed better to match the 
saccades' characteristics in your data. 
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Mind Games: The Worlds of 
Psychophysics and Virtual 
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Introduction and Learning Objectives 


So far in this textbook, you have explored the visual system in Chapter 8 and 
the use of eye trackers in Chapter 9, both of which are critical for 
understanding how we perceive and interact with our environment. These 
chapters laid the groundwork for appreciating the complex interplay between 
visual stimuli and perceptual responses. 

In this next chapter, we will explore how to integrate psychophysics and virtual 
reality (VR) and how these technologies can be harnessed to advance our 
understanding of sensory perception. We will discuss the principles of 
psychophysics, the benefits of VR in psychophysical research, and the various 
types of psychophysical tests that can be enhanced through VR. Furthermore, 
we will explore trends and future directions, including multisensory 
experiments, computational advancements, and ethical considerations. By the 
end of this chapter, you will be able to: 


Understand the basic concepts and methodologies of psychophysics 
and their application in VR environments. 

Identify and explain the psychophysical tests that can be conducted 
using VR technology. 

Discuss the advantages and potential challenges of integrating VR 


into psychophysical research. 

Explore the latest trends and future directions in psychophysics, in- 
cluding multisensory experiments and computational models. 
Recognize the ethical considerations involved in conducting psy- 
chophysical research in immersive VR settings. 


Introduction to Psychophysics 


Definition and Scope of Psychophysics 

Psychophysics is the scientific study of the relationship between physical 
stimuli and the sensations and perceptions they evoke. It aims to quantify the 
link between objective stimulus properties and subjective sensory experiences, 
enabling a deeper understanding of how we perceive the world around us. 


405 


Historical Overview 


The origins of psychophysics date back to the 
mid-19th century with the pioneering work of 
Gustav Fechner, who is often regarded as the 
father of psychophysics (Fig. 10.1 [208]) 
[209]. Fechner's law, which describes the 
logarithmic relationship between stimulus 
intensity and sensation, was a significant 
breakthrough. Other key contributors include 
Ernst Weber, known for Weber's Law, which 
quantifies the smallest noticeable difference in 
stimulus intensity, and S.S. Stevens, who 
developed the concept of Stevens' power law, 


addressing the relationship between stimulus 


: 3 ; : Figure 10.1: Gustav Theodore 
magnitude and perceived intensity [210]. See (1801-1887) [208] 


Fundamentals of 
Psychophysical Methods 


Basic Concepts 


Psychophysics explores the connection between stimuli and our sensory 
perceptions. Two essential concepts in this field are thresholds and sensitivity. 
The absolute threshold is the minimum intensity of a stimulus that can be 
detected. In contrast, the difference threshold (or just noticeable difference, 
JND) is the slightest change in stimulus intensity that can be detected. 
Sensitivity refers to the ability to detect these thresholds (Fig. 10.2 [211]). The 
psychometric function (Fig. 10.3 [212]) illustrates these relationships, showing 
how response probability changes with stimulus intensity, highlighting low 
and high-sensitivity regions [213]. 


Common Psychophysical Paradigms 


In psychophysical experiments, three primary methods are used to explore 
sensory thresholds and responses (Fig. 10.3 [214]). 
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Figure 10.2: The absolute threshold represents the weakest amount of a stimulus that a person 
can detect 50% of the time. Examples include seeing a candle flame 30 miles away on a cleat 
night, hearing a watch ticking 20 feet away, feeling a bee's wing falling on the cheek from 1 
cm, smelling one drop of perfume in a three-room home, and tasting one teaspoon of sugar in 
two gallons of water. [211] 


Method of Limit 

This approach involves gradually increasing or decreasing stimuli. For in- 
stance, a light might be dimmed or brightened step by step until the participant 
indicates whether they can or cannot detect it. This helps determine the point 
of detection or absolute threshold [215]. 

Method of Adjustment 

Participants have control over the stimulus intensity. They adjust it themselves 
until they can detect the stimulus. This method is often quicker and can provide 
immediate feedback on threshold levels [215]. 
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Method of Constant Stimuli 

This method randomly presents stimuli of varying intensities. The participant 
indicates whether they detect each stimulus. This comprehensive method 
allows for the creation of a detailed psychometric function that shows the 
probability of detection across different stimulus intensities [215]. 


Advantages of Using VR in Psychophysics 


Virtual Reality (VR) significantly enhances psychophysical research by 
offering a controlled and immersive environment. One of the key benefits is 
improved control over stimuli, allowing researchers to manipulate virtual 
environments with high precision and ensuring consistent and replicable 
experimental conditions [187]. Additionally, VR provides a strong sense of 
immersion, making experimental tasks more engaging and realistic for 
participants. Another advantage is the ability to recreate VR environments, 


response 


stimulus 
(independent variable, set by experimenter) 


Figure 10.3: The psychometric function illustrates the relationship between stimulus intensity 
(the independent variable set by the experimenter) and the response. The sigmoidal shape of 
the curve demonstrates regions of low and high sensitivity, showing how small changes in 
stimulus intensity can produce significant changes in response at intermediate intensities. This 
function is crucial for understanding absolute and difference thresholds in psychophysics. [214] 
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Infrared 
RF 


Wired 


VR hardware 


Figure 10.4: A detailed VR setup for psychophysics experiments. The system includes hard- 
ware components such as VR headsets (HMDs), controllers, lighthouses for positional track- 
ing, and wireless adapters, all interconnected via infrared, RF, and wired connections. The setup 
is managed by a central computer. The software workflow involves VR hardware integration 
through SteamVR, interfaced with Python (OpenVR), and controlled via a MATLAB GUI for 
precise experimental design and data analysis. [217] 


which can be exactly reproduced across different sessions and participants, 
ensuring the consistency and reliability of experimental conditions [216]. 

VR Technologies in Psychophysics 

VR technology in psychophysics integrates various hardware and software 
components to create immersive and precise experimental environments. 

The hardware (Fig. 10.4 [217]) comprises VR head-mounted displays 
(HMDs), motion tracking systems, haptic devices, and specialized controllers 
that enhance sensory feedback and interaction [218]. The setup often involves 
devices such as lighthouses for positional tracking, controllers for input, and 
wireless adapters for connectivity, all coordinated by a central computer. 

On the software side, VR platforms and simulation software, such as SteamVR 
and custom Python scripts, are tailored for psychophysical experiments. These 
tools allow researchers to design, control, and analyze sensory stimuli within 
virtual environments, providing detailed and replicable experimental data 
[218]. 
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Case Studies 


VR technology is increasingly utilized in psychophysics to investigate sensory 
perceptions and cognitive processes. Various case studies illustrate VR's 
diverse applications and benefits in this field. 

One notable case study involves using VR to examine the visual perception of 
surface materials. Researchers utilized VR to manipulate visual cues such as 
texture, glossiness, and reflectance properties, providing insights into how 
these factors influence material perception under different lighting conditions. 
This study highlighted the ability of VR to create controlled, immersive 
environments for detailed psychophysical experiments [219]. 

Another application of VR in psychophysics is in the realm of exposure 
therapy for anxiety and post-traumatic stress disorder (PTSD). VR exposure 
therapy allows patients to engage with virtual environments that simulate 
anxiety-inducing scenarios in a controlled and safe manner. This method has 
significantly reduced symptoms by enabling repeated and consistent exposure 
without the constraints of real-world settings [219]. 

Furthermore, VR has been used to investigate human behavior and cognitive 
functions. For example, the Unity Experiment Framework (UXF) leverages 
VR to study spatial cognition and navigation. By immersing participants in 
virtual mazes or landscapes, researchers can manipulate variables such as 
spatial layout and environmental cues to observe their effects on navigation 
and memory. This approach offers precise control over experimental 
conditions and the ability to measure responses accurately [219]. 

Training and education also benefit from VR's immersive capabilities. Hydro 
One and Avangrid have implemented VR training programs to enhance 
employees’ skills and safety procedures [220]. These programs simulate real- 
world scenarios, allowing trainees to practice responses to various situations 
without the risks associated with on-the-job training. The interactive and 
engaging nature of VR training has proven to increase retention and 
performance in trainees [219]. 

These case studies demonstrate the versatility and effectiveness of VR in 
advancing psychophysical research and applications, providing valuable 
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insights into sensory processing, cognitive functions, and therapeutic 
interventions. 


Types of Psychophysical Tests 
Visual Psychophysics 


Visual psychophysics studies the relationship between physical stimuli and the 
perceptual experiences they produce, focusing on how the visual system 
interprets and processes various aspects of light and color. One of the key areas 
of study is contrast sensitivity, which refers to the ability to detect differences 
in luminance between an object and its background. This ability is vital for 
activities like night driving, where low light conditions make it difficult to see 
objects unless they contrast sharply with their surroundings. High contrast 
sensitivity is also essential for reading, allowing for a clear distinction between 
text and background. Another critical area is color discrimination, which 
assesses an individual’s ability to differentiate between various hues. This is 
particularly important in professions requiring color accuracy, such as graphic 
design, quality control in manufacturing, or even medical diagnostics, where 
subtle color differences might indicate a condition or issue. Visual acuity, 
another comerstone of visual psychophysics, measures the clarity or sharpness 
of vision. It is fundamental for tasks that require attention to detail, such as 
reading fine print, recognizing faces, or driving. Visual acuity is often tested 
using standardized charts that measure how well someone can discern letters 
or symbols at a given distance.[221]. 


Auditory Psychophysics 

Auditory psychophysics explores the relationship between acoustic stimuli 
and auditory perception, investigating how the auditory system processes 
sound. A primary focus is determining auditory thresholds, the quietest sounds 
an individual can detect. These thresholds are crucial in diagnosing hearing 
impairments and are commonly tested using pure tone audiometry. Identifying 
these thresholds helps fit hearing aids and other assistive listening devices. 
Another important aspect of auditory psychophysics is sound localization, 
which determines where a sound originates in space. This skill is essential for 
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effective communication in noisy environments, as it allows individuals to 
focus on a speaker's voice amid background noise. Moreover, sound 
localization is vital for spatial awareness, helping people navigate 
environments safely and effectively, whether crossing a street or detecting 
potential hazards [222]. 


Olfactory and Gustatory Psychophysics 


Olfactory and gustatory psychophysics examine the senses of smell and taste, 
respectively, and their importance in daily life. Olfactory sensitivity involves 
detecting and identifying various odors critical for safety and quality of life. 
For instance, the ability to smell gas leaks, smoke, or spoiled food can prevent 
accidents and health risks. Additionally, olfactory sensitivity plays a 
significant role in flavor perception, as the sense of smell is closely linked with 
taste, contributing to the overall experience of food and beverages. On the 
other hand, Gustatory psychophysics focuses on understanding taste and 
distinguishing between different taste qualities—sweet, sour, bitter, salty, and 
umami. This ability is crucial for maintaining a balanced diet, as it influences 
food preferences and nutritional intake. Moreover, taste sensitivity is important 
in the culinary arts, where subtle flavor distinctions can impact the outcome of 
a dish. Both olfactory and gustatory sensitivities can diminish with age or due 
to certain medical conditions, affecting quality of life [223]. 


Somatosensory Psychophysics 


Somatosensory psychophysics investigates the sense of touch, including how 
the body perceives physical sensations such as pressure, vibration, pain, and 
temperature. Tactile sensitivity is a key study area, focusing on detecting 
delicate touch and texture. This sensitivity is crucial for tasks that require fine 
motor skills, such as writing, using tools, or manipulating small objects. It also 
plays a significant role in perceiving the physical properties of objects, such as 
softness, roughness, or slipperiness. Pain perception is another critical area 
involving the assessment of pain threshold (the minimum stimulus that causes 
pain) and pain tolerance (the maximum intensity of pain an individual can 
endure). Understanding pain perception is essential for diagnosing and treating 
chronic pain conditions and developing effective pain management strategies 
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in clinical settings. Temperature sensitivity examines the ability to detect 
temperature changes, which is necessary for thermal regulation and avoiding 
harmful environments. For example, sensing extreme heat or cold prevents 
burns, frostbite, or other injuries [224]. 


Current Trends and Future Directions in 
Psychophysics 


Integrating Multisensory Experiments 


Recent psychophysics research increasingly focuses on integrating multiple 
sensory modalities to comprehensively understand how they interact and 
influence perception. Traditional psychophysical experiments often isolated 
one sense, such as vision or hearing, to study its characteristics. However, real- 
world sensory experiences typically involve the simultaneous processing of 
information from multiple senses. For instance, flavor perception involves 
taste and smell, while navigating through an environment requires the 
integration of visual, auditory, and vestibular inputs. 

Fig. 10.5 [225] illustrates this multisensory approach by presenting two 
experiments with varying reliability conditions for visual, auditory, and tactile 
stimuli [225]. Experiment la includes conditions with low and medium 
auditory (LV-MA) and high visual and medium auditory (HV-MA) conditions. 
Experiment 1b expands this to three reliability conditions: high visual, medium 
auditory, and low tactile (HV-MA-LT); low visual, medium auditory, and high 
tactile (LV-MA-HT); and low visual, medium auditory, and low tactile (LV- 
MA-LT). These setups aim to understand how sensory inputs are weighted and 
integrated under varying reliability conditions [225]. 
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Experiment 1.a with two reliability conditions Experiment 1.b with three reliability conditions 
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Figure 10.5: Multisensory experiments from Mahani et al. showing varying reliability condi- 
tions for visual, auditory, and tactile stimuli. Experiment setups include combinations of high, 
medium, and low reliability to study sensory integration. [225] 


By designing experiments considering these multisensory interactions, 
researchers aim to uncover more accurate models of human perception and 
understand how the brain integrates and prioritizes different sensory inputs. 
This holistic approach can lead to new insights into sensory processing and its 
applications in virtual reality, augmented reality, and sensory rehabilitation. 


Ethical Considerations 


The immersive and often highly realistic nature of VR in psychophysical 
research introduces a range of ethical considerations. One primary concern is 
ensuring that participants give informed consent and fully understand the 
potential risks and benefits of participating in VR studies. VR is a complex and 
relatively new technology for many participants, which means that researchers 
must take extra steps to ensure that participants fully understand what VR 
entails. This includes explaining how the VR system works, the nature of the 
simulations they will experience, and the specific tasks they will be asked to 
perform. Additionally, researchers must communicate the potential benefits 
and risks of participating in VR studies. For example, while VR can provide 
valuable insights into cognitive processes or therapeutic benefits, there are 
risks, such as motion sickness, disorientation, or psychological discomfort. 
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Participants must be fully aware of these possibilities before consenting to 
participate in the study. It is crucial to emphasize that participation in VR 
research is voluntary, and participants can withdraw from the study at any time 
without penalty. Given the immersive nature of VR, which can sometimes be 
overwhelming, participants should feel empowered to stop the experience if 
they feel uncomfortable. VR environments can collect a wide range of data 
beyond what is typically gathered in traditional research settings. This includes 
detailed tracking of movements, gaze patterns, physiological responses (such 
as heart rate and galvanic skin response), and even voice and facial 
expressions. This data can provide deep insights into participant behavior and 
raise significant privacy concerns. Researchers must implement robust 
measures to protect the privacy of participants. This includes anonymizing the 
data wherever possible to prevent it from being traced back to individual 
participants. Data security protocols must be in place to protect against 
unauthorized access, breaches, or data misuse. Encryption and secure storage 
solutions are essential to maintaining data integrity and confidentiality. The 
extensive data collected in VR studies must be used responsibly. Researchers 
should be transparent about how the data will be used, who will access it, and 
for what purposes. Participants should be informed about any potential future 
uses of their data, including whether it might be shared with other researchers 
or used in subsequent studies [226]. 

Additionally, the highly immersive nature of VR can lead to various 
psychological effects, some of which may be adverse. For instance, 
participants might experience stress, anxiety, or even a sense of detachment 
from reality after prolonged exposure to certain VR environments. Scenarios 
that involve heights, confined spaces, or intense simulations can be particularly 
challenging for some individuals. To mitigate these risks, researchers should 
pre-screen participants for any conditions that might make them more 
vulnerable to adverse effects, such as a history of anxiety disorders, motion 
sickness, or vertigo. During the study, participants should be closely monitored 
for signs of distress, and protocols should be in place to immediately intervene 
if a participant experiences discomfort. After the VR experience, participants 
should undergo a thorough debriefing session to discuss their experience and 
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any lingering effects. Providing psychological support or referrals to 
counseling services may be necessary for participants who experience 
significant stress or anxiety as a result of the VR study. Given the relatively 
new use of VR in research, there is a need for comprehensive ethical guidelines 
that specifically address the challenges associated with VR. These guidelines 
should be developed in consultation with ethicists, psychologists, and other 
stakeholders to cover aspects such as informed consent, data privacy, and the 
psychological impact of VR. Ethical protocols for VR studies should be 
rigorously reviewed by Institutional Review Boards (IRBs) or ethics 
committees. These bodies should assess whether the study design adequately 
protects participants’ rights and well-being and whether the potential benefits 
of the research justify any risks involved. As VR technology and applications 
evolve, so should the ethical guidelines governing its research use. Ongoing 
ethical review and protocol updates are necessary to address new challenges, 
ensuring participant safety and well-being remain the top priority [227]. 
While short-term use of VR in controlled settings is generally safe, the long- 
term effects of repeated or prolonged exposure to immersive environments are 
not yet fully understood. Researchers should be cautious in designing studies 
that require extended use of VR and consider the potential cumulative effects 
on participants’ mental health and perception. VR’s ability to create highly 
realistic simulations presents additional ethical considerations. For instance, 
using VR to simulate traumatic events or highly stressful situations could have 
profound psychological impacts. Researchers must balance the need for 
realism with the ethical imperative to avoid causing harm. In cases where 
realistic simulations are necessary, extra care must be taken to screen 
participants and provide support. 

VR in psychophysical research presents exciting possibilities for 
understanding human behavior and cognition in immersive environments. 
However, it also introduces various ethical challenges that must be carefully 
managed to protect participants. Ensuring informed consent, safeguarding 
privacy, monitoring psychological impact, and adhering to strict ethical 
guidelines are all crucial components of conducting responsible and ethical 
VR research. As VR technology continues to advance, the ethical frameworks 
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surrounding its use in research must evolve accordingly, ensuring that the 
benefits of this powerful tool are realized without compromising the well- 
being of participants. 


Future Work 


Recent trends in psychophysics and VR are showing promising applications 
for individuals with autism spectrum disorder (ASD) and Parkinson's Disease. 
For individuals with ASD, VR provides controlled environments to practice 
social interactions and develop communication skills without the 
unpredictability of real-life situations. Studies have shown that VR can help 
improve social cognition and reduce anxiety in social settings for individuals 
with ASD [228], [229], [230]. Future work aims to refine these VR 
applications to cater to specific needs and to integrate real-time feedback 
mechanisms to enhance the effectiveness of these interventions. 

For Parkinson's Disease, VR is used to develop rehabilitation programs that 
help improve motor skills and balance. VR environments can simulate various 
physical activities and provide immediate feedback, assisting patients in 
practicing and improving their movements safely. Research has demonstrated 
that VR-based therapy can enhance motor function and reduce symptoms of 
Parkinson's Disease. Future directions involve integrating more personalized 
and adaptive VR systems that can adjust the difficulty level based on the 
patient's progress, ensuring a more tailored rehabilitation experience [229]. 
These advancements highlight VR's potential to create engaging and effective 
therapeutic environments, offering new hope for individuals with ASD and 
Parkinson's Disease. Continued research and development will lead to even 
more sophisticated and beneficial applications. 


Chapter 10: Summary 


Promising applications of VR in psychophysics include therapeutic 
interventions for Autism Spectrum Disorder (ASD) and Parkinson's Disease. 
For ASD, VR provides controlled environments to practice social interactions 
and develop communication skills, reducing anxiety and improving social 
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cognition. For Parkinson's Disease, VR rehabilitation programs help improve 
motor skills and balance, enhancing motor function and symptom reduction. 
Future work aims to refine VR applications with real-time feedback and 
adaptive systems, offering personalized therapeutic environments. These 
advancements will shape the future of psychophysics, providing new insights 
and effective treatments for sensory and cognitive disorders. 

Chapter 10 explored the integration of psychophysics and Virtual Reality 
(VR), highlighting how VR transforms our sensory perception understanding. 
Traditionally, psychophysical tests have been conducted in controlled, static 
environments to isolate specific sensory responses. VR has revolutionized this 
approach by providing dynamic, immersive environments replicating real- 
world conditions while maintaining experimental control. 

VR enhances psychophysical research by enabling complex, multi-sensory 
experiences that are otherwise impractical to recreate. Researchers can 
simulate diverse environments, examine sensory interactions, and tailor 
experiments to individual participants. Key trends include multisensory 
studies, advancements in computational power, and emerging ethical 
considerations in VR research. Promising VR applications in psychophysics 
include therapeutic interventions for Autism Spectrum Disorder (ASD) and 
Parkinson's Disease. For ASD, VR offers controlled environments for 
practicing social interactions, improving communication skills, and reducing 
anxiety. For Parkinson's Disease, VR-based rehabilitation programs help 
improve motor skills and balance, leading to better symptom management and 
quality of life. 

Looking ahead, the refinement of real-time feedback and adaptive systems 
within VR holds great potential for personalized therapeutic environments. 
These advancements will shape the future of psychophysics, offering new 
insights and effective treatments for sensory and cognitive disorders. 

Having explored the intersection of VR and psychophysics, we now focus on 
the auditory system. In the next chapter, you will learn about the auditory 
system, and we will explore how the auditory system processes sound, 
including pitch, loudness, and spatial location. Through this exploration, you 
will gain a deeper understanding of how hearing is measured and analyzed 
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using psychophysical methods and the implications this has for both research 
and clinical practice. 
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Chapter 10: Learning activities 


Learning Activity 10.1 
Brainstorming Psychophysics Paradigms 


Objective 


Students will collaboratively brainstorm and compile a 
comprehensive list of known psychophysics paradigms and descriptions of 
what each paradigm tests. 


Materials 


e Whiteboard or flip chart 

e Markers 

e Internet access (for research) 

e Computers or tablets 

e Note-taking materials (paper, pens, or digital tools) 


Time 
20-30 minutes 


Instructions 

Introduction (3 minutes): 

1.Introduce the activity and explain its goal: to brainstorm and compile a list 
of psychophysics paradigms and what they test. 

2.Briefly explain a psychophysics paradigm's importance in understanding 
sensory perception and cognitive processes. 

Individual Brainstorm (5 minutes): 

3. Instruct students to individually brainstorm and list as many psychophysics 

paradigms as possible. Encourage them to think about what each paradigm 

tests. 

Peer Share (5 minutes): 

4. Have students pair and share their lists, combining their knowledge to 

expand them. 
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Group Compilation (10 minutes): 

5. Form small groups of 4-5 students. Each pair will join another pair. 6. In 
these groups, students will combine their lists and organize the paradigms into 
a comprehensive list detailing what each paradigm tests. 7. Provide large 
sheets of paper or poster boards and markers for the groups to write down their 
combined lists. 

Class Presentation and Compilation (7-10 minutes): 

8. Each group will present their compiled list to the class. 9. As each group 
presents, compile a master list on the whiteboard or flip chart to avoid 
duplicates and include all unique paradigms. 

Conclusion (2 minutes): 

10. Summarize the key points and the final comprehensive list of 
psychophysics paradigms, emphasizing the variety and scope of what each 
paradigm tests. 11. Encourage students to reflect on the importance of these 
paradigms in psychophysical research and their applications. 


Some Examples of Psychophysics Paradigms: 


1. Signal Detection Theory (SDT) 

What it Tests: Assesses the ability to distinguish between signal (stimulus) 
and noise (non-stimulus) under conditions of uncertainty. It measures 
sensitivity (d') and response bias (criterion). 

2. Method of Limits 

What it Tests: Determines the threshold at which a stimulus is detected. It 
involves gradually increasing or decreasing the intensity of a stimulus until the 
subject can detect it. 

3. Method of Constant Stimuli 


What it Tests: Establishes the sensory threshold by randomly presenting 
stimuli of varying intensities and recording the detection frequency. 
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4. Method of Adjustment 


What it Tests: Allows the subject to control the intensity of a stimulus, 
adjusting it until it is barely detectable. This method finds the just-noticeable 
difference (JND) or absolute threshold. 


5. Two-Alternative Forced Choice (2AFC) 


What it Tests: Participants are presented with two options and must choose 
which contains the target stimulus. This method helps to reduce response bias 
and measure sensory discrimination. 


6. Staircase Procedure 


What it Tests: Adaptive method where the stimulus intensity is adjusted based 
on the subject's responses. If the subject detects the stimulus, the intensity is 
decreased; if not, it is increased. This method converges on the threshold level. 


7. Magnitude Estimation 


What it Tests: Participants assign numerical values to the perceived intensity 
of a stimulus. This method is used to study the relationship between stimulus 
intensity and perceived magnitude. 


8. Cross-Modal Matching 


What it Tests: Involves comparing the intensity of stimuli across different 
sensory modalities (e.g., matching the loudness of a sound to the brightness of 
a light). This method explores sensory integration and perception. 


9. Forced-Choice Staircase 


What it Tests: Combines forced-choice methodology with the staircase 
procedure. Participants are given multiple trials with varying stimulus 
intensities, and the intensity is adjusted based on correct or incorrect responses. 


10. Temporal Order Judgment (TOJ) 


What it Tests: Assesses the ability to perceive the order of two stimuli 
presented in quick succession. This paradigm is used to study temporal 
resolution and processing. 
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Il. Visual Search Task 


What it Tests: Measures how quickly and accurately a participant can find a 
target stimulus among a set of distractors. This paradigm tests attention and 
perceptual processes. 


12. Contrast Sensitivity 

What it Tests: Determines the minimum contrast needed to detect a pattern. 
This method is used to study visual acuity and sensitivity to contrast. 

13. Lateral Masking 


What it Tests: Examines the effect of surrounding stimuli (masks) on the 
perception of a target stimulus. This paradigm investigates spatial interactions 
and inhibitory processes in perception. 


Learning Activity 10.2 
Designing a Psychophysics Experiment 
Objective 


Students will design a psychophysics experiment related to 
their classroom project, present a functional block diagram, and write pseudo- 
code for the experiment. 


Materials 


e Computers or tablets 

e Drawing materials (paper, markers) or diagramming software 
e Note-taking materials 

e Reference materials (textbooks, research articles) 

e Presentation tools (PowerPoint, poster boards) 


Time 
50-60 minutes 
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Instructions 


Introduction (5 minutes): 


1. 


Introduce the activity and explain its goal: to design a psychophysics 
experiment related to the classroom project, create a functional block 
diagram, and write pseudocode. 

Briefly review the key components of a psychophysics experiment, 
functional block diagrams, and pseudocode. 


Brainstorming and Planning (10 minutes): 


ep 


Have students brainstorm ideas for their psychophysics experiment. 
Encourage them to think about what aspect of perception they want to 
study and how it relates to their classroom project. 

Ask students to outline the main components of their experiment, in- 
cluding stimuli, tasks, measurement methods, and expected outcomes. 


Designing the Experiment (15-20 minutes): 


5. 


Instruct students to create a functional block diagram of their experi- 
ment. This diagram should include all major components and the flow 
of the experiment. 

Provide examples of functional block diagrams if necessary and assist 
students as they work on their designs. 


Writing Pseudocode (10-15 minutes): 


7 


8. 


Have students write pseudocode for their experiment, detailing the se- 
quence of steps, data collection methods, and any computational pro- 
cesses involved. 

Provide examples of pseudocode and offer guidance as needed. 


Group Presentation Preparation (5 minutes): 


s} 


Allow students to prepare a brief presentation of their experiment 
design, including the functional block diagram and pseudocode. They 
should be ready to explain their choices and the overall flow of the 
experiment. 


Presentations and Feedback (15-20 minutes): 


10. Have each student present their experiment design to the class. 
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11. Encourage peer feedback and constructive criticism to help refine the 
experiments. 


Conclusion (5 minutes): 
12. Summarize the activity and highlight key points from the presenta- 
tions. 
13. Emphasize the importance of careful experimental design and the role 
of functional block diagrams and pseudocode in planning and execut- 
ing psychophysics experiments. 


@ 
PAs 
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Chapter 10: Lab introduction 


In this series of lab exercises, you will explore advanced techniques 
used in psychophysics research through practical applications using 
MATLAB. These labs will provide hands-on experience with virtual 
reality (VR) and Psychtoolbox, allowing you to deepen your 
understanding of how these tools can be applied in psychological 
experiments. 

In the first lab, you will be introduced to the use of VR in 
psychophysics research by interacting with a simple VR world using 
MATLAB. This exercise will guide you through loading a VRML 
(Virtual Reality Modeling Language) world, opening it in a VR 
viewer, and interacting with basic objects within the VR 
environment. This will help you understand how VR can be utilized 


to create controlled, immersive environments for psychological 


experiments. 

In the second lab, you will explore Psychtoolbox and apply your 
knowledge through a practical task. Specifically, you will implement 
a working memory task inspired by the visual working memory 
capacity experiment conducted by Vogel and Machizawa (2004). 
This exercise will provide you with experience in setting up and 
conducting a psychological experiment using Psychtoolbox, 
enhancing your skills in experimental design and data analysis in the 
field of psychophysics. 
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Chapter 10: Lab Example 1 


Overview 


This lab exercise aims to introduce students to virtual reality 


(VR) in psychophysics research by interacting with a simple 
VR world using MATLAB. The exercise will guide students through loading 
a VRML (Virtual Reality Modeling Language) world, opening it in a VR 
viewer, and interacting with basic objects within the VR environment. 


Requirements 


e MATLAB software 

e = Virtual Reality Toolbox 

e VRML world file (simple_world.wrl’) 
e Provided MATLAB code 


Part 1: Setting Up the VR Environment 

Step 1: Preparing MATLAB 

Ensure the MATLAB Virtual Reality Toolbox is installed and configured on 
your system. 

Step 2: Creating the VRML World 


Create a simple VRML world file (simple world.wrl’). This file should 
contain basic objects such as a table and a ball. 


simple_world.wrl: 
~**wrl 
VRML V2.0 utf8 
WorldInfo { 

title "Simple World" 


} 


Background { 
skyColor [0.8 @.8 1.0] 
} 


Transform { 
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translation 2 @ @ 
children [ 
Shape { 
geometry Box { size 4 8.2 4 } 
appearance Appearance { 
material Material { diffuseColor 8.8 @.1 0.1 } 
} 


} 
Transform { 
translation @ 1 @ 
children [ 
Shape { 
geometry Sphere { radius 0.5 } 
appearance Appearance { 
material Material { diffuseColor i i 1 } 


Part 2: Interacting with the VRML World in MATLAB 
Step 3: Loading the VRML World 


Use the provided MATLAB code to load and open the VRML world ina VR 
viewer. MATLAB Code: 


% Load the VRML world 
vrWorld = vrworld('simple world.wrl'); 


% Open the VRML world 
open(vrWorld) ; 


% Create a viewer window for the VRML world 
vrFigure = vrfigure(vrWorld) ; 
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Step 4: Running the Code 


1. 
2. 


4. 


Save the provided VRML code in a ‘simple_world.wrl' file. 

Open MATLAB and navigate to the directory containing 
‘simple world.wrl’. 

Copy and paste the MATLAB code into the MATLAB command window 
or script file. 

Run the MATLAB code to load the VRML world and open the VR viewer. 


Step 5: Exploring the VR Environment 

Interact with the VRML world using the VR viewer controls. The VR world 
should look like the image provided, featuring a simple setup with a red table 
and a white ball on top of it against a light blue background. 

Observe the simple objects (table and ball) and their properties. 


e Hand and Ball interaction 
File View Viewpoints Navigation Rendering Recording Help » 
Initial view GS Examine OJ KrP ABQ Aa ee ma» 


rill view T=0.00 Examine Pos:{0.00 2.00 17.001 Dir-10.00 0.00 -1.001 


Figure 10.6: Hand and Ball Interaction 
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Part 3: Experimentation 
Step 6: Modifying the VRML World 


Modify the ‘simple world.wrl’ file to change the properties of the objects 
(e.g., color, size, position). Reload the VRML world in MATLAB to see the 
changes. 


Step 7: Recording Observations 
Note how different properties of the objects (such as size and color) affect your 
perception in the VR environment. What are the potential applications of VR 


in psychophysical experiments, such as studying the perception of size, color, 
and spatial relationships? 


Conclusion 


This lab exercise introduced you to the basics of using VR in psychophysics 
research. You learned how to create a simple VRML world, load it into 
MATLAB, and interact with it using a VR viewer. By modifying the VR 
environment and observing the changes, you gained insights into the potential 
applications of VR for studying sensory perception. 


Further Reading 


e MATLAB Documentation on Virtual Reality Toolbox: [MATLAB VR 
Toolbox] (https://www.mathworks.com/help/s13d/) 

e VRML and X3D _ Specifications: [Web3D Consortium] 
(https://www.web3d.org/ ) 


A 
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Chapter 10: Lab Example 2 


In this lab, we will explore Psychtoolbox and apply our 
knowledge through a practical task. Specifically, we will 
implement a working memory task inspired by the visual 


working memory capacity experiment conducted by Vogel 
and Machizawa (2004). You can access the original research article at 
https://www.nature.com/articles/nature02447. 
Psychtoolbox is a collection of free software tools that facilitates designing 
and implementing precise and complex visual and auditory stimuli for 
psychological experiments. It is widely used in psychology, neuroscience, and 
vision research. Here are some key features and components of Psychtoolbox: 
e Platform Compatibility: Psychtoolbox is primarily used with MATLAB and 
GNU Octave, making it accessible to users who prefer different 
programming environments. 
e Stimulus Presentation: It allows for creating and controlling visual and 
auditory stimuli with high temporal and spatial precision. This is crucial for 
experiments requiring exact timing and synchronization. 


Graphics and Sound: Psychtoolbox includes functions for generating and 
manipulating graphics, images, and sounds. This allows researchers to 
create complex visual displays and auditory sequences. 

e Hardware Interaction: It supports interaction with various hardware devices 
such as eye trackers, response boxes, and other input/output devices, 
enabling comprehensive experimental setups. 

e Open Source: Being an open-source toolkit, Psychtoolbox is freely 

available and can be modified to suit specific research needs. 


Community Support: A strong community of users and developers 
contribute to the toolbox, provide support, and share scripts and solutions. 
Psychtoolbox is particularly powerful in experiments that require precise 
control over stimulus timing and presentation, such as studies in visual 
perception, auditory processing, and reaction time measurement. 

It is available for both MATLAB and Octave. This powerful tool offers a range 
of commands that simplify the process of creating experiments. For more 
information about the available functions, visit Psychtoolbox Documentation. 
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After installing Psychtoolbox in MATLAB, you can begin designing your 
experiments. In this instance, we will use an implementation provided by 
Marvin Theiss. 


Working memory experiment 


The visual working memory capacity experiment conducted by Vogel and 
Machizawa in 2004 aimed to explore the limitations and neural mechanisms 
underpinning visual working memory. Participants in the study were presented 
with arrays of colored rectangles and tasked with remembering their colors and 
locations. Following a brief delay, they were shown a second array and had to 
determine if it was identical to the first or if one rectangle had changed color. 
The number of rectangles varied to assess memory capacity. EEG recordings 


arrow presentation 
200 ms 


memory array 
400 ms 


retention interval 


900 ms 


test array 
2,000 ms 


Figure 10.7: Experiment Task 


were used to monitor neural activity, particularly focusing on the contralateral 
delay activity (CDA). 
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The study found that participants’ accuracy in detecting changes declined as 
the number of items increased, with a notable drop when the set size exceeded 
three to four items, indicating a memory capacity limit of about 3-4 items. 
Correspondingly, the amplitude of the CDA increased with the number of 
items up to this capacity limit, beyond which it plateaued, suggesting a fixed 
neural resource for visual working memory. This research highlighted that 
visual working memory is constrained by a capacity limit and established the 
CDA as a neural marker for the number of items held in memory, offering 
significant insights into the cognitive and neural mechanisms of working 
memory limitations. Once Psychtoolbox is installed, you must run the 
WorkingMemoryCapacity.m script. The code will first prompt participants to 
complete an identification form. 


prompt = {'Participant ID (1 - 999):', 
"Please enter your sex (m/w/d):', .. 
"Please enter your year of birth: '}; 


This allows experimenters to keep track of the data collected during the 
experiment. Additionally, the experiment provides options for adjusting 
specific parameters. You can change the number of squares or objects to 
remember and the total number of trials, determining how often the task will 
be repeated. In psychophysics, repeating the task is important because some 
effects may be too subtle to detect in just a few trials; however, repeating the 
task allows these effects to become more apparent when averaged over a larger 
number of trials. 


% Set number of colored squares per hemifield 

% NOTE: Parameters used by Vogel & Machizawa (2004): 1, 
By Bp thy Gy Hy HG 

nSquares = 4; 


% Number of trials (excluding practice trials) 

% NOTE: Vogel & Machizawa (2004) conducted 24@ trials in 
each experiment 

nTrials = 240; 
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% Number of practice trials 
nPracticeTrials = 10; 


SOA stands for “Stimulus Onset Asynchrony.” It refers to the time interval 
between one stimulus's onset and another. SOA is commonly used in 
experimental psychology, particularly in studies investigating perception, 
attention, and reaction times. By manipulating the SOA, researchers can 
examine how the timing of stimuli affects cognitive processes and behavioral 
responses. For example, in a visual attention experiment, varying the SOA 
between a cue and a target can help determine the speed and efficiency of 
attentional shifts. In our case, this variable makes the human subject unable to 
predict when the next stimulus will happen. 


% Set range to be used for the SOA (values are the ones 
used by Vogel & 

% Machizawa (2004) ) 
Duration.stimOnsetAsyncMinSecs 
Duration.stimOnsetAsyncMaxSecs = 


@.3; % in secs 
0.4; % in secs 


Some other parameters deal with the timing of the experiment, which is crucial 
when getting certain results for working memory. 


% Set remaining timing parameters for the experiment 
(again, values are the 
% ones used by Veet & Machizawa (20@4)) 


Duration.arrowSecs = Q.2; % in secs 
Duration.memoryArraySecs = @.1; % in secs 
Duration.retentionIntervalSecs = 0.9; % in secs 
Duration.testArraySecs = 2; % in secs 


Maintaining a consistent viewing distance in working memory tasks is crucial 
for several reasons. Firstly, it ensures that the visual angle of the stimuli 
remains constant, allowing for uniform perception across trials and 
participants. This consistency is essential for controlling the perceptual aspects 
of the task, ensuring that all stimuli appear the same size and are processed 
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similarly by the visual system. A proper viewing distance also guarantees that 
stimuli are visible and in focus, preventing blurriness or difficulty 
distinguishing the items, which could increase cognitive load and lead to 
errors. Controlling the viewing distance also minimizes the need for large eye 
movements, reducing variability in how stimuli are processed and 
remembered. Large eye movements could cause some stimuli to fall outside 
the central part of the visual field, where visual acuity is highest, thus 
impairing memory encoding. 

Furthermore, a consistent viewing distance helps preserve the spatial 
relationships between stimuli, which is crucial in tasks where spatial 
configuration and relative positions are important for performance. 
Standardizing viewing distance across studies also allows for better 
comparison of results, enabling researchers to draw broader conclusions about 
visual working memory. Lastly, it helps control for variations in depth 
perception, ensuring that all participants experience the stimuli in a similar 
two-dimensional plane without unintended depth cues. Overall, maintaining 
an appropriate viewing distance ensures uniform perception, reduces 
variability, and enhances the reliability and validity of working memory 
research. 


% (Orthogonal) distance from eye to screen in mm 

% NOTE: This depends heavily on the setup (chair, desk, 
Laptop vs. external 

% monitor, etc.) that's being used. With my setup, I 
measured the 

% following distances (using a height-adjustable desk 
and desk chair that 

% are properly adjusted to me): 


% - W/ Laptop screen (MacBook Pro 16"): 55@ mm 
%  - W/ external monitor (Dell U4@210W 46" attached to 


Ergotron HX): 65@ mm 
viewingDistanceMM = 550; % in mm 
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Visual angle 


Another important factor is the size of the objects to be retained in working 
memory. In this experiment, they subtend a visual angle of 0.65 degrees. 


% NOTE: Vogel & Machizawa (2004) used squares with a 
SUZE Of, OnObie eX OOS a. 
squareSizeVA = 0.65; % in degrees of visual angle 


Visual angle measures how large an object appears to an observer based on its 
size and distance from the observer. It is important in fields like vision science 
and experimental psychology because it helps quantify how much of the visual 
field an object occupies. Here’s how visual angle is calculated: ### Formula 
for Visual Angle The visual angle (8) in degrees can be calculated using the 
formula: 


d 
6=2 t (|) 10.1 
X arctan aD (10.1) 


Where: 

- 0 = Visual angle in degrees 

- d = Diameter or height of the object (or stimulus) 

- D = Distance from the observer to the object Steps for Calculation 


1. Measure the Object’s Size (d): Determine the object's size or stimulus in 
units such as centimeters or inches. 

2. Measure the Viewing Distance (D): The distance from the observer’s eyes 
to the object or stimulus. This is often done in centimeters or inches. 

3. Apply the Formula: Substitute the values for d and D into the formula to 
find the visual angle. 
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Example Calculation 


Suppose you have an object 10 cm in diameter (d), viewed from a distance of 
100 cm (D). 
1. Convert the measurements into the formula: 


@ =2xaret (TE) = 2 t (=) (10.2) 
= arctan | 99) = arctan | 755 : 
2. Calculate the visual angle: 
6 = 2 x arctan (0.05) = 2 x 2.86° = 5.72° (10.3) 


So, the visual angle of the object is approximately 5.72 degrees. 


Subject responses 


Participants are asked to respond if they notice a change in the square 
configuration, indicating whether the pattern is identical or different. 
Psychtoolbox uses specific code to facilitate response mechanisms, such as the 
keyboard. In this case, pressing the letter ‘J’ signifies that the pattern is 
identical while pressing ‘F’ indicates that it is different. 


% Set keys 

% NOTE: The space bar will be used by participants to 
navigate through 

% instructions. To indicate a decision, participants 
will press either 'J' 

% (indicating identical arrays) or 'F' (indicating 
different arrays). 

% Finally, the escape key can be used to prematurely end 
the experiment. 

Key.space = KbName('SPACE" ); 

Key.identical = KbName('J'); 
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Key.different = KbName('F'); 
Key.escape = KbName('ESCAPE" ); 


The response is then logged after every trial of the experiment. 


% 7.2 Store participant's response 
trials.Response(iTrial) = response; 
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Figure 10.8: Data in Excel File 


The responses, along with the data from each trial, are then saved in an Excel 
file. 

The package from Marvin Theiss also includes a procedure for analyzing the 
data generated from the experiment. This script processes the CSV file created 


ParticipantID Items Trials HitRate FalseAlarmRate Capacity 


1 4 239 0.87395 0.025 3.3958 


Figure 10.9: Trial Variables 
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during the experiment and calculates the hit rate, false alarm rate, and working 
memory capacity. In working memory tasks, capacity is typically calculated 
using the measure “K,” which represents the number of items a participant can 
hold in working memory. This measure is often derived from a change 
detection task where participants are briefly shown an array of visual stimuli, 
such as colored rectangles. After a short delay, a second array is presented, and 
participants must determine if it is the same as the first array or if one item has 
changed. The number of items in the arrays, or set size, varies across trials. 
The proportion of correct responses (hit rate) and incorrect responses when no 
change occurs (false alarm rate) are recorded to calculate K. The formula for 
calculating working memory capacity is K = S x (H — F), where S is the set 
size, H is the hit rate, and F is the false alarm rate. For example, with a set size 
of 4 items, a hit rate of 0.75, and a false alarm rate of 0.20, the capacity K 
would be 4x (0.75 — 0.20) = 2.24 x (0.75 — 0.20) = 2.24 x (0.75 — 
0.20) = 2.2. This means the participant can accurately maintain and process 
about 2.2 items in their visual working memory. This calculation is essential 
for understanding individual differences in cognitive abilities and comparing 
performance across different populations or experimental conditions. 
In our case, the capacity is approximately 3.4, which means that, on average, 
the participant (us) can hold and accurately maintain about 3.4 items in their 
visual working memory. This measure indicates the maximum number of 
items a person can effectively store and manipulate in their mind over a short 
period. 
Implications of a Capacity of 3.4 

6. High Performance: A capacity of 3.4 is relatively high, suggesting that 

participants can retain and process visual information. 
7. Comparison to Typical Capacities: This value is above the commonly 
cited average working memory capacity of 3-4 items, indicating that par- 


ticipants perform at or slightly above average regarding their visual work- 
ing memory abilities. 
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Experimental Context: This capacity is context-dependent and can be 
influenced by various factors, such as the stimuli's nature, the retention 
interval's duration, and the specific task demands. 

Cognitive Implications: Higher working memory capacity is often asso- 
ciated with better performance in tasks that require attention, problem- 
solving, and learning. It suggests that participants can efficiently manage 
and utilize multiple visual information simultaneously. A working 
memory capacity of approximately 3.4 means that participants can sim- 
ultaneously hold and accurately recall around 3.4 visual items, reflecting 
a robust capacity for visual information processing. This measure pro- 
vides a valuable understanding of their cognitive abilities and can inform 
further research or practical applications where working memory plays a 
crucial role. The most common capacity for visual working memory, of- 
ten cited in psychological and cognitive neuroscience research, is typi- 
cally around 3-4 items. This estimate is based on numerous studies inves- 
tigating the limits of working memory using various tasks and methodol- 
ogies. 


Evidence about capacity (K) 


10. 


11. 


12. 


Classic Studies: Early research by George Miller in 1956 suggested that 
the average capacity of short-term memory is about 7 + 2 items. However, 


more recent studies focusing specifically on visual working memory have 
found that this capacity is smaller, generally around 3-4 items. 
Empirical Findings: Many experiments using change detection tasks, 
similar to the ones described by Vogel and Machizawa (2004), consist- 
ently find that participants can hold approximately 3-4 items in visual 
working memory before their performance declines. 

Contralateral Delay Activity (CDA): Research using neural measures 
such as CDA supports this capacity limit. The CDA amplitude increases 
with the number of items held in memory up to about 3-4 items, after 
which it plateaus, indicating a maximum capacity. 


We invite you to complete this task to determine your working memory 
capacity. 
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Chapter 11 


Tuning In: Exploring the 
Marvels of The Auditory System 
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Introduction to the Auditory System 


The auditory system is one of the most remarkable and specialized sensory 
systems in the human body, playing a crucial role in our ability to interact with 
the world. This system is responsible for detecting, transmitting, and 
processing sound, allowing us to experience a vast array of acoustic stimuli— 
ranging from the rumbling of thunder and the roar of a crowd to the subtle 
rustling of leaves and the intricate harmonies of a symphony. The auditory 
system’s ability to discern these diverse sounds with incredible precision is 
essential for communication, environmental awareness, and the appreciation 
of music and language. The auditory system is a complex network that begins 
with the ear's anatomy and extends to the intricate pathways within the brain. 
In addition to the natural hearing processes, this chapter will explore the role 
of auditory nerves and how they connect the ear to the brain, forming the 
pathways that carry sound information. 

Furthermore, we will examine the function of hearing assistance technologies, 
such as hearing aids and cochlear implants, designed to support individuals 
with hearing impairments. These technologies have advanced significantly 
over the years, offering new possibilities for those who experience hearing 
loss, helping to restore the ability to perceive and enjoy the rich world of 
sound. By understanding how this system operates, we gain insights into the 
marvels of human hearing and how technology can enhance it for those in 
need. By the end of this chapter, you will be able to: 


Understand the in-depth anatomy of the ear and the mechanism of 
hearing. 
Identify the importance of the auditory nerves and their role in 
hearing. 


Understand the different forms of hearing loss and their various 
impacts on the auditory system. 

Explain the principles behind hearing assistance techniques such as 
cochlear implants. 
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Overview of Auditory System Function 


The auditory system’s primary function is to convert sound waves into 
electrical signals for our brain to interpret. Besides simply detecting sound, the 
auditory system also interprets the various characteristics of sound, such as 
pitch, loudness, and timbre. The overall system of hearing is accomplished 
through several steps: the outer ear collects sound waves, which travel through 
the middle ear via mechanical vibrations; the inner ear transforms sound waves 
into fluid waves, and the brain generates neural signals. The anatomy of each 
ear component and how they collectively achieve this function will be 
explored in detail in this chapter. Additionally, complex signal transmissions, 
processing, and interpretation in the brain will be discussed in detail. 


The Ear Anatomy 


The human ear is an intricate and highly specialized organ responsible for the 
critical sense of hearing. It is divided into three main sections: the outer ear, 
middle ear, and inner ear. Each section plays a distinct and vital role in 
converting sound waves from the environment into electrical signals that the 
brain can interpret as sound. The seamless cooperation between these sections 
allows us to perceive a wide range of sounds, from the faintest whisper to the 
loudest explosion. The basic structure of the human ear is shown in Fig. 11.1 
[231]. 


Outer, Middle, and Inner Ear Anatomy 


Outer Ear 

The outer ear serves as the gateway to the auditory system, capturing sound 
waves and funneling them toward the deeper structures of the ear. It comprises 
two primary components: the auricle (or pinna) and the external auditory me- 
atus (ear canal). The auricle, the visible part of the ear, is uniquely shaped to 
help gather sound waves from the environment and direct them into the ear 
canal. Its curved, cartilaginous structure enhances our ability to detect the di- 
rection from which sounds are coming, a crucial aspect of spatial awareness. 
The external auditory meatus is a narrow passageway lined with skin, hair, and 
glands that produce earwax, which helps protect the ear by trapping dust and 
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Figure 11.1: The structure of the human ear [231] 


other foreign particles. The outer ear's primary function is to collect and direct 
sound waves to the tympanic membrane, or eardrum, which marks the bound- 
ary between the outer and middle ear. The tympanic membrane is a thin, cone- 
shaped membrane that vibrates in response to incoming sound waves, initiat- 
ing the process of sound transmission deeper into the ear. 


Middle Ear 

The middle ear is an air-filled cavity located behind the eardrum. It plays a 
critical role in amplifying and transmitting sound vibrations from the outer ear 
to the inner ear. The middle ear contains three tiny bones known as the ossicles: 
the malleus (hammer), incus (anvil), and stapes (stirrup). These bones are the 
smallest in the human body, yet they are essential for hearing. When sound 
waves cause the tympanic membrane to vibrate, these vibrations are 
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Figure 11.2: The Inner Ear [232] 


transferred to the malleus, which is attached to the eardrum. The malleus then 
transmits the vibrations to the incus, which, in turn, passes them to the stapes. 
The stapes, the final bone in the chain, are connected to the oval window, a 
membrane-covered opening that leads to the inner ear. 

The primary function of the ossicles is to amplify the sound vibrations. As the 
vibrations pass through the ossicles, they are concentrated and strengthened to 
overcome the impedance mismatch between the air-filled middle ear and the 
fluid-filled inner ear. This amplification ensures sound waves are efficiently 
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transmitted into the cochlear fluid. The Eustachian tube connects the middle 
ear to the nasopharynx (upper part of the throat). This small canal helps 
equalize air pressure on both sides of the eardrum. This equalization is 
important for maintaining proper ear function and preventing discomfort 
during activities such as swallowing, yawning, or changes in altitude. 


Inner Ear 

The inner ear is a complex and highly sensitive structure responsible for 
hearing and balance. It contains two main components: the cochlea, which is 
responsible for listening, and the vestibular system, which is responsible for 
balance. The general inner ear anatomy is shown in Fig. 11.2 [232]. The 
cochlea is a spiral-shaped, fluid-filled organ that resembles a snail shell. It is 
the primary organ of hearing, where mechanical sound vibrations are 
transformed into electrical signals that the brain can interpret. The cochlea is 
divided into three fluid-filled compartments: the scala vestibuli, scala media, 
and scala tympani. The scala vestibuli and scala tympani contain perilymph, 
similar to cerebrospinal fluid. In contrast, the scala media contains endolymph, 
a fluid with a unique ionic composition essential for hair cell function. 

At the heart of the cochlea lies the basilar membrane, a flexible structure that 
runs along the length of the cochlea and plays a key role in sound processing. 
Embedded within the basilar membrane are thousands of tiny hair cells, which 
are the sensory receptors of the auditory system. As sound vibrations travel 
through the fluid in the cochlea, they cause specific regions of the basilar 
membrane to vibrate. Different sound frequencies stimulate various areas of 
the basilar membrane, with high-frequency sounds affecting the base of the 
cochlea and low-frequency sounds affecting the apex. The basilar membrane 
movement causes the hair cells to bend, triggering the conversion of 
mechanical energy into electrical impulses. These impulses are then 
transmitted to the brain via the auditory nerve, where they are interpreted as 
sound. 

Although this chapter focuses primarily on the auditory system, it is important 
to briefly mention the vestibular system, which is also housed within the inner 
ear. The vestibular system comprises the semicircular canals and otolithic 
organs, which detect changes in head position and motion. This system plays 
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a crucial role in maintaining balance and spatial orientation. While not covered 
in detail here, the vestibular system’s role in balance and coordination is 
essential for overall sensory integration. 


Sound Characteristics 


Now that you have a general understanding of the anatomy of the ear and a 
brief description of how hearing works let's discuss in greater detail the 
mechanism of how sound waves are transmitted to the brain in the form of 
speech or music for us to hear. First, we need to understand sound waves and 
the properties of sound that our anatomy allows us to discern. Several physical 
properties characterize sound, such as frequency, amplitude, and timbre. 


Frequency 

The frequency of a sound wave is usually measured in cycles per second or 
Hertz. A frequency of 1 Hz indicates one complete sound wave cycle per 
second; this may also be called an oscillation. The frequency is directly related 
to the perceived pitch. Higher frequencies correspond to higher pitches. The 
human auditory system can detect sounds from approximately 20 Hz to 20,000 
Hz, although our sensitivity is highest between 500 Hz and 5,000 Hz. A graph 
of a sine wave showing a constant frequency and amplitude is shown in Fig. 
3. In this image, if the frequency, or the number of oscillations, increased, this 
would be perceived as a higher pitch sound. If it decreased, it would be 
perceived as a lower pitch sound. 


Amplitude 

The amplitude of a sound wave is measured in decibels (dB). We perceive 
amplitude as loudness. In a sound wave, the amplitude is the wave’s height, 
where larger amplitudes produce louder sounds. In Fig. 3, if the amplitude 
increased in height, the sound would be more intense. 


Timbre 

Timbre refers to a sound's quality; this enables us to distinguish between dif- 
ferent sound sources. For example, when a musician plays the violin with a 
singer, our ear can discriminate the violin sound and identify its source from 
the singer, even when both sounds are produced at the same pitch and loudness. 
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This is arguably one of the more fascinating concepts of human hearing and is 
essential for identifying complex sounds. 


Mechanism of Hearing 


We can explore the hearing mechanism in more detail with a fundamental 
understanding of sound waves and auditory system anatomy. Sound waves 
enter the ear canal and cause the eardrum to vibrate—the ossicles transmit and 
amplify vibrations. The ossicles enable proper transmission of sound waves to 
vibrations of the cochlear fluid by impedance matching [233]. Additionally, 
these ossicles focus sound on a concentrated area of the cochlea, the oval 
window. When the cochlear fluid waves move, the basilar membrane also 
moves. The projections of the inner hairs, called stereocilia, are sensitive to the 
movement of the basilar membrane. This movement of the basilar membrane 
bends the stereocilia, which causes the opening of ion channels and 
depolarization, converting the fluid wave into an electrical signal. The outer 
hairs of the basilar membrane also change in size, which amplifies and fine- 
tunes the cochlea's response. 

The basilar membrane is sensitive to various frequencies and will respond 
differently along its length to each frequency. The base of the cochlea is stiffer, 
narrower, and more sensitive to high-frequency sounds. The apex of the 
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Figure 11.3 An example sine wave demonstrating amplitude and frequency. 
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membrane is wider, more flexible, and more sensitive to low-frequency 
sounds. This varied sensitivity to different sound frequencies is referred to as 
tonotopic organization. Specific nerve endings positioned along the basilar 
membrane enable us to distinguish the magnitude and range of frequencies, up 
to around 20,000 Hertz (Hz) [234]. 


Auditory Nerves and Processing 


The vestibulocochlear nerve, or cranial nerve 8, is responsible for the ability 
to hear and balance. The cochlear nerve component transmits the electrical 
signals from the cochlea to the brain for sound to be processed and interpreted. 
As many as 30,000 auditory nerve fibers comprise our vestibulocochlear nerve 
[235, p. 9]. This allows us to discern the range of piano notes and voices we 
can hear. As we will discuss later, damage to this nerve can significantly impact 
our auditory processing. 


Cochlear Nuclei 


The electrical impulses from the auditory nerves are sent to the cochlear nuclei, 
which are in the rhomboid fossa of the brainstem. The cochlear nuclei are the 
first major relay station where signals are refined. Here, sound information is 
split into parallel processing streams that separately handle different features 
of the sound, such as pitch, timing, and intensity. The dorsal cochlear nucleus 
(DCN) integrates auditory signals with other sensory inputs, crucial for 
understanding sound in a noisy environment. In contrast, the ventral cochlear 
nucleus (VCN) processes the timing and intensity of sound with high 
precision, allowing for the accurate localization of sound sources. 


Superior Olivary Complex 


As the auditory signals travel upward from the cochlear nuclei, they reach the 
Superior Olivary Complex (SOC) in the brainstem's pons. The SOC is crucial 
for binaural hearing. This is the process where our brain uses input from both 
ears to determine the location of a sound source. The SOC is in the first place 
in the auditory pathway, where information from both ears converges. Audi- 
tory signals from the right ear crossover are forwarded to the contralateral side 
of the brain; however, some research suggests that ipsilateral signals also play 
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a significant role [236]. The collection of steps before the brainstem is consid- 
ered the peripheral auditory system, and the brain stem and the brain are the 
central auditory nervous system. 


Inferior Colliculus 


The auditory signals are relayed from the SOC to the inferior colliculus. The 
inferior colliculus is a pair of major integration centers for auditory 
information. It receives inputs not only from the SOC and cochlear nuclei but 
also directly from the auditory cortex. The inferior colliculus is responsible for 
further refining sound processing handling complex aspects such as temporal 
patterns, spatial location, and specific pitch and intensity modulation features. 
This area acts as a filter and relay station, ensuring that only important auditory 
information is transmitted to the higher brain centers. 


Medial Geniculate Nucleus 


The information from the inferior colliculus is then sent to the thalamus, 
specifically the medial geniculate nucleus of the thalamus. The medial 
geniculate nucleus has several 
subnuclei that play a role in 
processing auditory 
information. It’s primary 
function is to act as the main 
relay from the inferior 
colliculus to the auditory 
cortex, Al, located in the 
temporal lobe. 


Temporal Lobe 


Primary Auditory Cortex 
This primary auditory cortex 


(A1) and the associative or sec- 


ondary cortex are collectively — Figure 11.4 An AI generated image of the brain with 
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location of the primary auditory cortex. The primary auditory cortex is the first 
cortical area to receive auditory input and is responsible for the basic interpre- 
tation of the auditory signals. Here, the auditory information is mapped tono- 
topically, with different regions of the auditory cortex responding to other fre- 
quencies. This highly precise tonotopic organization allows for the detailed 
frequency discrimination we discussed earlier. This is fundamental to perceiv- 
ing speech, music, and other complex sounds. 

In addition to frequency mapping, the auditory cortex is involved in the initial 
stages of sound processing, such as detecting the presence of a sound, deter- 
mining its pitch, and spatial location. These basic processing tasks proceed to 
the more complex auditory processing by the associative cortex. 


Secondary Auditory Cortex 

The secondary auditory cortex (A2) lies adjacent to Al in the temporal lobe 
and is critical in higher-order sound processing. This region begins to interpret 
complex auditory signals, such as recognizing speech or music and 
distinguishing different types of sounds. Two primary streams are within the 
secondary cortex: the ventral and dorsal pathways. The ventral pathway plays 
a significant role in sound identification and recognition. The dorsal pathway 
is involved in spatial localization of sounds. 


Wernicke’s Area 

Wernicke's area is within the temporal lobe. This site is responsible for our 
higher-order speech comprehension. Wernicke’s area plays a vital role in pro- 
cessing the meaning of spoken and written language. Essentially, this area 
links the sound of words to their associated meanings. Disruptions to this area 
can result in Wernicke’s aphasia, severely impairing speech comprehension. 
Interestingly, speech production itself can remain unaffected. 


Hearing Loss 


There are three major types of hearing loss: conductive, sensorineural, and 
mixed [237]. These three types of hearing loss affect the auditory systems 
differently and have distinct pathophysiology. The next sections will discuss 
treatment methods for each type of hearing loss. 
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Conductive Hearing Loss 

Conductive hearing loss occurs when there is an issue with conducting sound 
waves. Therefore, this hearing loss primarily affects the outer and middle ears. 
Some common causes of conductive hearing loss include structural 
abnormalities or trauma to the eardrum, ear infections, objects, and fluid. 
These problems prevent sound from being transferred from the outer and 
middle ear to the inner ear. 


Sensorineural Hearing Loss 

Where conductive hearing loss encompasses problems with the outer and 
middle ears, sensorineural hearing loss occurs with issues with the inner ear. 
This hearing loss occurs when problems with the cochlea or auditory nerves 
lead to the brain. It is the most common type of hearing loss and is usually 
permanent. Some examples of sensorineural hearing loss include aging, loud 
noise exposure, and illnesses. Age-related hearing loss is specifically referred 
to as presbycusis. 


Mixed Hearing Loss 

Mixed hearing loss includes a combination of conductive and sensorineural 
loss. This means there is damage to both the outer/middle and inner ear. For 
example, say you had an ear infection followed by extremely loud noise 
exposure. These two combined issues lead to mixed hearing loss and are the 
most difficult to address. 


Genetic Factors 

Genetic mutations can play a role in congenital and progressive hearing loss 
like those discussed above. Mutations of specific genes can cause sensorineu- 
ral hearing loss. Recent advances in genetic testing show promise for targeted 
interventions for individuals with these genetic mutations. 

Trauma-Induced Hearing Loss 


Although traumatic incidents may still fall under the scope of conductive, sen- 
sorineural, or mixed hearing loss, they deserve discussion. Various types of 
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Figure 11.5: Image of an ear with cochlear implant [238] 


trauma can influence the method and magnitude of the subsequent hearing 
loss. Traumatic brain injury, for example, may lead to disturbance of the audi- 
tory cortex. Damage to this region or the surrounding area can range from 
buzzing and roaring sensations to complete deafness. Earlier in this chapter, 
we briefly discussed Wernicke’s aphasia, a specific type of traumatic-induced 
hearing loss disorder. A different kind of traumatic-induced hearing loss in- 
cludes noise-induced hearing loss. This sensorineural hearing loss can be 
caused by prolonged exposure to loud noise or sudden, damaging acoustic 
trauma. Direct trauma to the eardrum in the form of punctures will also con- 
tribute to hearing loss. There are various degrees to which trauma-induced 
hearing loss can be treated or mitigated. 
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Hearing Loss Treatment 
Hearing Aids 


Hearing aids are small electronic devices designed to assist individuals with 
hearing loss by amplifying sounds. These devices are particularly beneficial 
for those experiencing mild to moderate hearing loss, whether due to conduc- 
tive issues, such as blockages or damage in the outer or middle ear, or senso- 
rineural problems, such as damage to the inner ear or auditory nerve. The basic 
components of a hearing aid include a microphone, amplifier, and speaker. The 
microphone picks up sound from the environment, which is then converted 
into electrical signals. The built-in amplifier increases the strength of these 
signals, making them louder and easier to hear. Finally, the amplified signals 
are delivered to the ear through a speaker. 

Modern hearing aids are highly sophisticated, offering a range of customizable 
features that allow them to be precisely tuned to the user's specific hearing 
profile. This customization is crucial because hearing loss often affects differ- 
ent frequencies unevenly. For example, some individuals may have difficulty 
hearing high-pitched sounds, while others may struggle with low-frequency 
noises. Advanced hearing aids can be programmed to amplify specific frequen- 
cies more than others, thereby providing a tailored listening experience that 
enhances speech comprehension and reduces background noise. Additionally, 
modern hearing aids often include directional microphones, noise reduction, 
and wireless connectivity to smartphones and other devices, making them ver- 
satile tools for managing hearing loss in various environments. 


Cochlear Implants 


Cochlear implants represent a more advanced technological solution for 
individuals with severe to profound sensorineural hearing loss, typically 
resulting from damage to the hair cells within the cochlea; unlike hearing aids, 
which amplify sound, cochlear implants work by directly stimulating the 
auditory nerve, bypassing the damaged parts of the ear. This approach enables 
individuals who might otherwise be unable to hear to regain a degree of 
auditory perception. 
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The cochlear implant system consists of both external and internal 
components. The external part, usually worn behind the ear, includes a 
microphone and sound processor that captures and converts sound into digital 
signals. These signals are then transmitted to a receiver implanted under the 
skin behind the ear. The internal receiver sends the signals to a series of 
electrodes surgically implanted into the cochlea. These electrodes directly 
stimulate the auditory nerve fibers, allowing the brain to perceive sound. While 
cochlear implants do not restore normal hearing, they can significantly 
improve speech understanding and environmental awareness, enabling users 
to participate more fully in everyday activities. Fig. 11.5 [238] shows a 
diagram of what a cochlear implant can look like. 


Ethical Considerations of Auditory Treatments 

While cochlear implants offer life-changing benefits for many individuals, 
their use also raises several ethical considerations, particularly within the Deaf 
community. For some, auditory treatments are seen as a medical miracle that 
can restore a sense of hearing to those who have lost it. However, others argue 
that auditory treatments represent an attempt to "fix" deafness, which they 
view not as a disability but as a unique cultural identity. This perspective is 
particularly strong within the Deaf community, where sign language is 
celebrated as a rich and fully developed means of communication, and Deaf 
culture is embraced as a valuable and distinct way of life. 

One of the primary ethical concerns centers around using auditory treatments 
in young children, especially infants who have not yet developed language or 
cultural identity. Parents and medical professionals often advocate for early 
implantation to maximize the potential for the child to develop spoken 
language skills. However, others argue that this decision may impose a 
hearing-centric view on the child before they are old enough to choose their 
identity and communication preferences. Another ethical consideration 
involves the potential impact on the Deaf community. As more individuals opt 
for auditory treatments, there is concern that Deaf culture and sign language 
may be marginalized or devalued. Advocates for Deaf culture emphasize the 
importance of preserving and promoting sign language as a legitimate and vital 
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means of communication, regardless of whether an individual has a cochlear 
implant. 

There is also the question of access and equity. Auditory treatments are 
expensive and require significant follow-up care, including speech therapy and 
regular adjustments to the device. This can create disparities in access, with 
only those with sufficient financial resources or healthcare coverage 
benefitting from the technology. Additionally, the long-term success of 
auditory treatments depends on consistent and intensive auditory training, 
which may not be available to all recipients, particularly in low-resource 
settings. 

In conclusion, while auditory treatments have the potential to significantly 
improve the quality of life for individuals with severe hearing loss, they also 
bring to light important ethical questions about identity, culture, access, and 
the definition of disability. Individuals, families, and healthcare providers must 
carefully weigh these considerations when making decisions about cochlear 
implantation. 


Neuroengineering Applications 


Neuroengineering has paved the way for innovative approaches to auditory 
rehabilitation. The previous fundamentals discussed in this chapter are 
essential for any engineer looking to further these treatments and technologies. 
As these technologies advance, so does our understanding of the auditory 
system. The development of cochlear implants and hearing aids has 
tremendously impacted those with hearing loss. Current research explores how 
to enhance these treatments to target other pathways and improve sound 
quality and speech recognition, especially in noisy environments. Research 
methods utilizing MATLAB and Psychtoolbox enable the accessibility of this 
type of research to many researchers of various experience levels. 


MATLAB and Psychtoolbox 


As you have seen from the previous sections in this chapter, MATLAB is a 
powerful tool for conducting various experiments and data processing. 
Psychophysics Toolbox, also known as Psychtoolbox, is a popular suite of 
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functions for MATLAB and can perform auditory and visual experiments 
[239]. The ease of use of this tool within MATLAB provides a vast array of 
simulation, analysis, and experimental design applications that can be used to 
conduct research in the auditory system. 


Other Experimental Methods 


Aside from the applications of MATLAB and Psychtoolbox, commercial- 
grade techniques such as the auditory brainstem response (ABR) and 
otoacoustic emissions tests (OAE) are frequently used in medical applications 
to screen for hearing loss. The OAE test can measure the response of the 
cochlea through auditory stimulation. The ABR test utilizes the principles of 
EEG that you learned earlier in this text and involves measuring the brain’s 
electrical activity in response to sound stimuli. 


Chapter 11: Summary 


The auditory system is very complex, enabling us to perceive and interpret the 
sounds around us. Understanding the anatomy, function, and pathophysiology 
is important for neuroengineers. Advances in hearing loss treatments, such as 
hearing aids and cochlear implants, are made possible by the research 
conducted in neuroengineering. Using the information presented in this 
chapter and the preceding chapters, you are armed with the knowledge to 
understand and conduct your research that can progress future technology and 
treatments. 
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Chapter 11: Learning Activities 


Learning Activity 11.1 


Connecting Chaos Theory and Cochlear 
Prostheses through a Mind Map 


Objective 


Students will independently explore the relationship between chaos theory and 


cochlear prostheses by creating a mind map, fostering an understanding of 


complex systems and their applications in medical technology. 


Materials Needed 


e Large sheets of paper or whiteboards 


e Markers or pens 


e Reference materials on chaos theory and cochlear prostheses (e.g., 
textbooks, articles, diagrams) 


Activity Outline 


1. Introduction (10 minutes) 


(2) 


Provide a brief overview of chaos theory, explaining its 
principles and significance in understanding complex, 
dynamic systems. 

Introduce cochlear prostheses, describing their function, 
components, and role in aiding individuals with hearing loss. 
Explain the purpose of the activity: to draw connections 
between the seemingly disparate concepts of chaos theory 
and cochlear prostheses through a mind map. 


2. Individual Mind Map Creation (30 minutes) 


(e) 


Each student will create a mind map on a large sheet of 
paper or whiteboard. 

They should place "Chaos Theory" and "Cochlear 
Prostheses" as the central nodes and branch out from these 
nodes to explore subtopics and connections. 

Encourage students to think creatively about how principles 
of chaos theory (such as sensitivity to initial conditions, 
nonlinearity, and complex, unpredictable behavior) might 
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relate to the design, functionality, or performance of cochlear 
prostheses. 

o Students should also consider how cochlear prostheses 
operate within the complex environment of the human 
auditory system, which can exhibit chaotic properties. 

3. Reflection and Note-Taking (10 minutes) 

o After completing their mind maps, students should reflect on 

their work and take notes on the following: 
« Key connections they identified between chaos 
theory and cochlear prostheses. 
« Any challenges they faced while drawing the mind 
map and how they overcame them. 
« Insights or new understandings they gained through 
the activity. 
4. Class Discussion and Reporting (10 minutes) 

o Each student will present their mind map to the class, 
explaining the connections they identified and the reasoning 
behind them. 

o After each presentation, allow time for questions and 
comments from the class to foster a deeper understanding 
and exploration of the topic. 

5. Conclusion (5 minutes) 

o Summarize the key points discussed during the class 
presentations, highlighting any particularly insightful or 
unique connections made. 

o Discuss the broader implications of applying chaos theory to 
understanding and improving medical technologies like 
cochlear prostheses. 

o Encourage students to continue exploring interdisciplinary 
connections in other areas of science and technology. 


Assessment: 


e Participation: Ensure that each student actively participates in the 
mind map creation and class discussion. 

e Creativity and Depth: Evaluate each student’s mind map for creativ- 
ity, depth of thought, and the ability to draw meaningful connections 
between chaos theory and cochlear prostheses. 
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e Presentation: Assess each student's presentation for clarity, coher- 
ence, and engagement with the class. 


Follow-Up: 


e Assign a research project where students investigate another medical 
technology through chaos theory, creating a detailed report or presen- 
tation on their findings. 

e Plan a guest lecture or webinar with a professional in biomedical en- 
gineering or applied mathematics to provide real-world insights into 
the connections between chaos theory and medical technologies. 


Learning Activity 11.2 


Final Activity: Creating a Concept Map of 
Book Concepts 


Objective 


Students will consolidate their understanding of all the concepts covered in the 
book by creating a comprehensive concept map illustrating how these concepts 
are interconnected. 


Materials Needed 


e Large sheets of paper or whiteboards 

e Markers or pens 

e Sticky notes (optional) 

e Reference materials, including notes, textbooks, and previous assign- 
ments 


Activity Outline 


1. Introduction (5 minutes) 
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e Explain the purpose of the activity: to create a concept map that 
connects all the concepts covered in the book to date. 

e Emphasize the importance of understanding the relationships be- 
tween different concepts for a holistic understanding of the subject 
matter. 

2. Preparation (10 minutes) 

e Allow students to gather their notes, textbooks, and other refer- 
ence materials. 

e Briefly review key concepts covered in the book, providing a list 
or a quick overview if necessary. 


1. Creating the Concept Map (30 minutes) 

o Each student will create a concept map on a large sheet of 
paper or whiteboard. 

o Students should start by placing the main theme or title of 
the book at the center of their map. 

o From the central theme, students will branch out to include 
all major concepts, sub-concepts, and details. Encourage 
them to use connecting lines, arrows, and labels to show 
relationships and dependencies between concepts. 

o Students can use different colors or symbols to categorize 
different types of concepts or to highlight particularly 
important connections. 

o Sticky notes can be used for ideas that might need 
rearranging as the map develops. 

2. Reflection and Note-Taking (5 minutes) 
o After completing their concept maps, students should reflect 
on their work and take notes on the following: 

" Key insights gained from creating the map. 

= Any connections between concepts that were 
previously unclear but became apparent during the 
mapping process. 

= Areas where they feel they need further review or 
understanding. 

3. Class Discussion and Reporting (20 minutes) 
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o Each student will present their concept map to the class, 
explaining the connections they identified and the overall 
structure of their map. 

o After each presentation, allow time for questions and 
comments from the class. This will help reinforce the 
material and provide different perspectives on connecting 
concepts. 

4. Conclusion (5 minutes) 

o Summarize the key points discussed during the class 
presentations. 

o Highlight common themes and unique insights from the 
concept maps. 

o Discuss the value of creating concept maps for 
understanding and retaining complex information. 

o Encourage students to use concept maps as a study tool in 
future learning. Learning activity 11.3.Learning activity 11.3 


Learning Activity 11.3 


Summary Activity: Storytelling to Connect 
Three Chapters 


Objective 


Students will synthesize and integrate concepts from three book chapters by 
crafting a compelling and engaging original story, which they can present in- 
dividually or in teams. 


Materials Needed 


e Paper and pens/pencils 

e Computers or tablets for writing and research (optional) 

e Reference materials, including notes, textbooks, and previous assign- 
ments 


Activity Outline 


1. Introduction (5 minutes) 
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e Explain the purpose of the activity: to create a compelling and en- 
gaging story that connects concepts from three chapters of the 
book. 

e Emphasize the importance of creativity and storytelling in under- 
standing and integrating complex ideas. 

Selection of Chapters (5 minutes) 


o Have students, individually or in teams, select three chapters 
from the book that they will connect through their story. 

o Encourage them to choose chapters with distinct but 
potentially related concepts to create an interesting narrative. 

Brainstorming and Planning (10 minutes) 

o Students will brainstorm ideas for their story, thinking about 
how the concepts from the selected chapters can be 
interwoven into a cohesive narrative. 

o They should plan the basic outline of their story, including 
characters, setting, plot, and how the key concepts will be 
integrated. 

Story Writing (30 minutes) 

o Students will write their story, ensuring it includes clear 
connections to the three selected chapters. 

o Encourage creativity and originality, and remind students to 
make their stories engaging and compelling. 

o Teams can divide writing tasks among members but should 
ensure the final story is cohesive. 

Reflection and Note-Taking (5 minutes) 
o After completing their story, students should reflect on their 
work and take notes on the following: 

= How did they integrate the concepts from the three 
chapters into their story? 

«Any challenges they faced during the storytelling 
process and how they overcame them. 

« Insights or new understandings they gained about 
the chapters through storytelling. 

Class Presentation and Discussion (25 minutes) 

o Each individual or team will present their story to the class, 
explaining how it connects the three chapters and the 
concepts within them. 
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o After each presentation, allow time for questions and 
comments from the class to foster a deeper understanding 
and appreciation of the creative connections made. 

6. Conclusion (5 minutes) 

o Summarize the key points discussed during the class 
presentations. 

o Highlight particularly creative and effective stories and 
explain why they stood out. 

o Discuss the value of storytelling in understanding and 
integrating complex ideas. 

o Encourage students to use storytelling to learn and 
communicate complex concepts in the future. 


Assessment 


e Participation: Ensure each student actively participates in the story 
creation and class presentation. 

e Creativity and Engagement: Evaluate each story for creativity, en- 
gagement, and originality. 

e Integration of Concepts: Assess how well the story integrates and 
connects the concepts from the three chapters. 

e Presentation: Evaluate each presentation for clarity, coherence, and 
engagement with the class. 


Follow-Up 


Assign a reflective essay where students analyze how storytelling helped them 
better understand and integrate the material and how they might use this 
technique in other subjects. 

Plan a storytelling workshop where students can practice and refine their 
storytelling skills, focusing on integrating academic concepts into engaging 
narratives. 
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Chapter 11: Lab introduction 


In this series of lab exercises, you will explore auditory perception 
and the effects of hearing loss using MATLAB. These labs are 
designed to provide hands-on experience with the Psychophysics 
Toolbox and MATLAB scripting, enhancing your understanding of 
auditory system functions and auditory testing methodologies. 

In the first lab, you will use MATLAB's Psychophysics Toolbox 
functions to perform an auditory test. By analyzing the hearing test 
results of an individual who has undergone tympanoplasty 
procedures, you will apply your knowledge of the auditory system 


to identify any abnormal results. This exercise will help you 


understand how auditory tests are conducted and interpreted in 
research. 

In the second lab, you will simulate hearing loss using MATLAB. 
You will visualize the results using time-domain plots and frequency 
spectra by generating sine wave tones at various frequencies and 
applying a low-pass filter to simulate hearing loss. Additionally, you 
will listen to the generated tones before and after the simulation to 
gain an auditory comparison. This lab will provide practical insights 
into hearing loss's impact on auditory perception. 
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Chapter 11: Lab Example 1 


Overview 


In this chapter, you learned about the Psychophysics Toolbox 
functions and how they can be used in MATLAB to conduct 
research. We will utilize one of these Psychtoolbox functions for the laboratory 


exercise to perform an auditory test. Of note is that the author of this chapter 
underwent two tympanoplasty procedures to reconstruct both eardrums. This 
was a result of complications from ear tubes as an infant. Using the knowledge 
you gained in this chapter and your newfound understanding of the auditory 
system, try to look for any abnormal results from the MATLAB hearing test 
performed by the author. 


Objective 


Use the Psychtoolbox functions in MATLAB to perform a rudimentary hearing 
test to examine the frequency range you can detect. Plot the frequency range 
in MATLAB. 


MATLAB Setup 


First, you should already have MATLAB installed on your computer. Next, go 
to Psychtoolbox and download the Psychtoolbox files: 


http://psychtoolbox.org/download.html#installation 


Installing Psychtoolbox is straightforward; installation instructions are 
provided in the link above. After downloading the Psychtoolbox folder, go to 
it in MATLAB and type “SetupPsychtoolbox” into the MATLAB command 
line. Done! 


MATLAB Code 


We will be utilizing the  Psychtoolbox function called 
“ExtendedToneReactionTest(). This function allows us to generate tones at 
different frequencies. The code, located on the GitHub link for this chapter, 
initializes the Psychtoolbox sound driver, generates tones at various 
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frequencies, plays those frequencies at three-decibel (dB) ranges, and 
measures response. 


Laboratory Results and Discussion 


Run the code and hit the spacebar (or any key) when you hear a tone. When 
the author performed this lab, he utilized a high-quality headphone system and 
listened in a quiet room. However, it should be mentioned that the decibel 
setting in the code is relative to the computer's volume setting and not 
reflective of the actual decibel level. Fig. 11.4 shows the test result from the 
author. 


Frequency Respgnse Test Results 
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Figure 11.4: Frequency Response Test Results of Male 


The green circles on the graph represent a response, and the red x represents 
no response. Next, let’s observe the results from a healthy adult female 
volunteer and compare. 

If we look at the frequency and decibel response, there is a notable difference 
in the higher-frequency range. Notice how the author could not hear any 
10,000 Hz tones and could only hear the 7,500 Hz tone at 70dB. The female 
volunteer could listen to both frequencies at 50dB and 70 dB. 
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Frequency Response Test Results 
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Figure 11.5: Frequency Response Test Results of female volunteer 
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Chapter 11: Lab Example 2 


Overview 


In this lab, we will be simulating hearing loss using 
MATLAB. The MATLAB script generates sine wave tones 
at various frequencies, simulates hearing loss using a low- 
pass filter, and visualizes the results using time-domain plots and frequency 
spectra. The script also plays the generated tones before and after simulating 
hearing loss to provide an auditory comparison. 


Requirements 


e MATLAB 

e Working speakers 

e Audio Toolbox MATLAB 
e Signal Processing Toolbox 


Steps 

First, you want to navigate to the GitHub page for this textbook. Here is the 
link 

: https://github.com/bddupre92/Neurobook BME UND 


After you are in Github, navigate to Chapter 11 Lab Example 2, and you will 
find a MATLAB file named chapter example.m. You will want to open that 
file in MATLAB and open it as a new script. Look at the code given. Can you 
recognize any functions you already know of? Do you see some that you don’t 
know? Here is a list of the main functions used in this MATLAB script. 


Main Functions Used 
sin 
e Description: Generates a sine wave. 
e Usage: signal = 0.5 * sin(2 * pi * f * t); 
e Purpose: To create the sine wave tones at specified frequencies. 


butter 
e Description: Designs a Butterworth filter. 
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abs 


Usage: [b, a] = butter(6, cutoff_freq/(fs/2)); 
Purpose: To design a low-pass filter to simulate hearing loss. 


Description: Applies a digital filter to a signal. 
Usage: signalHearingLoss = filter(b, a, signal); 
Purpose: To apply the low-pass filter to the generated sine waves. 


Description: Plays a sound. 
Usage: sound(signal, fs); 
Purpose: To play the generated sine wave tones. 


Description: Pauses execution for a specified number of seconds. 
Usage: pause(duration + 0.5); 
Purpose: To allow the sound to finish playing before proceeding. 


Description: Returns the length of a vector. 
Usage: length(combinedSignal); 
Purpose: To verify the lengths of the combined signals. 


Description: Creates a 2D line plot. 
Usage: plot((0:length(combinedSignal)-1)/fs, combinedSignal); 
Purpose: To plot the time-domain waveforms of the signals. 


Description: Computes the Fast Fourier Transform (FFT). 
Usage: fftCombinedSignal = fft(combinedSignal); 
Purpose: To compute the frequency spectrum of the signals. 


Description: Returns the absolute value. 


Usage: amplitude = abs(fftCombinedSignal)/N; 
Purpose: To compute the magnitude of the FFT. 
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xlim 
e Description: Sets the x-axis limits for the current axes. 
e Usage: xlim([0, max(frequencies)* 1.2]); 
e Purpose: To limit the x-axis to the relevant frequency range. 


xlabel, ylabel, title 
e Description: Labels the x-axis, y-axis and sets the plot's title. 
e Usage: xlabel('Time (s)');, ylabel(‘Amplitude’);, title('Time-Domain 
Waveform of Combined Signal (Normal Hearing)'); 
e Purpose: To add labels and titles to the plots. 


subplot 
e Description: Creates axes in tiled positions. 
e Usage: subplot(2, 1, 1); 
e Purpose: To create multiple plots in a single figure. 


e Description: Displays text or variables. 

e Usage: disp('Length of combinedSignal:’); 

e Purpose: To display messages and values in the command window. 

This script has four main steps that generate sounds at different frequencies, 
and then a low pass filter is used to model how hearing loss affects our ability 
to hear those sounds. 
The script then plots time domain signals and a composite frequency spectrum 
and combines them with the hearing loss signals to compare them. Finally, you 
will hear what the sounds would sound like for someone experiencing hearing 
loss. Here’s how the script does this. 


Step 1: Generate Sounds at Different Frequencies 


Description 


This step generates sine wave tones at specified frequencies and applies a low- 
pass filter to simulate hearing loss. The original and filtered signals are 
combined and played. 


Functions Used: 


1. sin: Generates the sine wave tones. 
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e signal = 0.5 * sin(2 * pi * f* t); 
2. butter: Designs a Butterworth low-pass filter. 
e = [b, a] = butter(6, cutoff_freq/(fs/2)); 
3. Filter: The low-pass filter is applied to the generated sine waves. 
e signalHearingLoss = filter(b, a, signal); 
4. sound: Plays the generated sine wave tones. 
e sound(signal, fs); 
5. pause: Pauses execution to allow the sound to finish playing. 
e pause(duration + 0.5); 


Step 2: Plot Time-Domain Signals 


Description 
This step plots the time-domain waveforms of the combined signals for both 
normal hearing and simulated hearing loss. 


Functions Used: 

1. plot: Creates a 2D line plot of the time-domain signals. 
e —plot((0:length(combinedSignal)-1)/fs, combinedSignal); 

2. xlabel, ylabel, title: Labels the x-axis y-axis, and sets the plot title. 
e xlabel('Time (s)');, ylabel(‘Amplitude’);, title(’Time-Domain 

Waveform of Combined Signal (Normal Hearing)'); 

3. subplot: Creates axes in tiled positions for multiple plots in a single 
figure. 
e = subplot(2, 1, 1); 


Step 3: Composite Frequency Spectrum 


Description 
This step computes the combined signals' Fast Fourier Transform (FFT) and 
plots the frequency spectra for normal and simulated hearing loss. 
Functions Used: 
1. fft: Computes the Fast Fourier Transform of the signals. 
e fftCombinedSignal = fft(combinedSignal); 
2. abs: Computes the magnitude of the FFT. 
e amplitude = abs(fftCombinedSignal)/N; 
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plot: Creates a 2D line plot of the frequency spectra. 

e = plot(f, amplitude); 

xlabel, ylabel, title: Labels the x-axis y-axis, and sets the plot title. 

e =xlabel('Frequency (Hz)');, ylabel('‘Amplitude');, title((Composite 
Frequency Spectrum of Combined Signal (Normal Hearing)'); 

xlim: Sets the x-axis limits for the current axes. 

e =xlim([0, max(frequencies)*1.2]); 

subplot: Creates axes in tiled positions for multiple plots in a single 

figure. 

e = subplot(2, 1, 1); 


Step 4: Play Tones Again at New Amplitudes to Simulate 
Hearing Loss 
Description 


This step plays the combined signal with simulated hearing loss to provide an 
auditory comparison of how the tones would sound after applying the hearing 
loss filter. 


Functions Used: 


sound: Plays the combined signal with hearing loss. 
e sound(combinedSignalHearingLoss, fs); 


2. pause: Pauses execution to allow the sound to finish playing. 


e pause(length(combinedSignalHearingLoss)/fs + 1); 
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Then you will want to save the script and then click Run. Ensure you have 
your speakers on your device and the volume is up. You hear the sounds two 
times before the script is finished running. You should notice that as the 
frequencies get higher, they get lower. The type of hearing loss simulated in 
this script is sensorineural hearing loss. Sensorineural hearing loss occurs due 
to damage to the inner ear's cochlear or auditory nerve pathway. It is the most 
common type of permanent hearing loss. The causes of this type of hearing 
loss can range and include gaining, exposure to loud noise, genetic factors, and 
diseases or infections. The main symptom of this hearing loss is loss of higher 
frequencies. With the loss pass filter used in the script, we can attenuate the 
higher frequencies, mimicking the characteristics of sensorineural hearing 
loss. You will notice that the script also generates two figures. Let's start with 
Figure 1, the Time Domain Signals. Fig. 11.6 is what it looks like. 
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Figure 11.6: Time Domain Wave forms for Normal Hearing vs Hearing Loss 
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The figure displays the time-domain waveforms of two combined audio 
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Figure 11.7: Composite Frequency for Normal Hearing vs Hearing Loss 


signals: one representing normal hearing and the other simulating 
sensorineural hearing loss. The top plot shows the waveform for normal 
hearing, where the amplitude remains consistent across the entire duration of 
approximately 10 seconds. This indicates that the generated sine wave tones, 
which are concatenated, maintain their integrity without any attenuation. In 
contrast, the bottom plot illustrates the waveform for the signal with simulated 
hearing loss. Here, the amplitude diminishes significantly over time, 
particularly after the initial few seconds. The result is a noticeable decrease in 
the signal's amplitude, especially for the higher frequencies, which manifests 
as a progressively weaker waveform. This visual representation effectively 
highlights the impact of sensorineural hearing loss on audio signals, 
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demonstrating how higher frequencies are diminished, leading to a loss of 
clarity and intensity in the sound. 

Now let’s look at Fig. 11.7, the Composite Frequency Spectrum. Here is what 
it looks like. The figure illustrates the composite frequency spectra of two 
combined audio signals: one for normal hearing and the other simulating 
sensorineural hearing loss. In the top plot, representing normal hearing, there 
are distinct and relatively uniform peaks at the frequencies where the original 
sine wave tones were generated—specifically at 100 Hz, 200 Hz, 300 Hz, 400 
Hz, 500 Hz, 600 Hz, 700 Hz, 800 Hz, 900 Hz, and 1000 Hz. This indicates that 
each tone is preserved with its original intensity, reflecting an ideal, 
undistorted signal where all frequency components are present as expected. 
In contrast, the bottom plot, representing the signal with simulated hearing 
loss, shows that the peaks at lower frequencies (100 Hz, 200 Hz, 300 Hz, 400 
Hz, and 500 Hz) remain prominent. However, the peaks at higher frequencies 
(above 500 Hz) are significantly attenuated or almost nonexistent. This 
attenuation is due to applying a low-pass filter, which mimics the effect of 
sensorineural hearing loss by reducing the sensitivity to higher frequencies. 
The comparison between the two plots demonstrates the impact of 
sensorineural hearing loss on the frequency content of the audio signal, 
effectively visualizing the loss of higher-frequency components. This is the 
conclusion to this lab example. We encourage you to visit our GitHub page for 
more practical lab examples. 
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Chapter 12 


The Battle Frontier: Fighting 
Neurological Disorders 
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ee Introduction and Learning Objec- 
ives 


Neurological disorders encompass various conditions that affect the nervous 
system, including the brain, spinal cord, and peripheral nerves. These disorders 
can significantly impact the lives of individuals, families, and society, leading 
to profound challenges in daily functioning and overall well-being. By the end 
of this chapter, you will be able to: 


Recognize and describe the key characteristics and symptoms of 
major neurological disorders, including Alzheimer’s disease, 
Parkinson’s disease, Autism Spectrum Disorder (ASD), Multiple 
Sclerosis (MS), Dementia, Huntington's disease, and Amyotrophic 
Lateral Sclerosis (ALS). 

Understand the physiological explanations underlying each 
neurological disorder, including the roles of genetic and 
environmental factors in their development and progression. 

Gain insights into how advanced technologies, such as imaging 
techniques, wearable devices, AI, and telemedicine, are used to 
diagnose, treat, and manage these disorders. 

Understand the social and economic implications of neurological 
disorders, including the burden on healthcare systems, and 
recognize the importance of early intervention and support in 
alleviating this burden and improving the lives of those affected. 
Learn about emerging trends and future research directions, 
including neuroprotective therapies, personalized medicine, and the 
potential of new technologies to transform neurological care. 


The burden of neurological disorders is immense, not only due to the direct 
healthcare costs but also the indirect costs, such as lost productivity and 
reduced quality of life. As our understanding of these disorders has evolved, 
so has the technology available to diagnose, treat, and manage them. Recent 
advancements offer hope for better patient outcomes and an improved quality 
of life for those affected. As we delve into the profiles of these major 
neurological disorders, we will explore their distinct characteristics, 
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underlying mechanisms, and innovative technological interventions that are 
changing the landscape of treatment and management. Let us begin with an in- 
depth look at these conditions, starting with Alzheimer’s disease. 


Detailed Disorder Profiles 


Alzheimer's Disease: A Complex Neurodegenerative Disor- 
der 


Alzheimer's disease (AD) is a progressive and irreversible neurological 
disorder that affects memory, thinking, and behavior. It is the most common 
form of dementia, accounting for 60-80% of dementia cases. AD is 
characterized by a decline in cognitive function, including memory loss, 
language difficulties, and problem-solving impairments [240]. As the disease 
progresses, individuals with AD may experience changes in personality, mood, 
and behavior, leading to a loss of functional abilities and independence [241]. 


Pathophysiological Mechanisms 


The exact causes of AD are still not fully understood, but research has 
identified several key pathophysiological mechanisms. One of the hallmark 
features of AD is the accumulation of amyloid-beta plaques outside neurons, 
which disrupts communication between brain cells and leads to cell death. 
Additionally, the formation of tau tangles inside neurons disrupts the transport 
system and contributes to neuronal damage. Brain atrophy, particularly in the 
hippocampus and temporal lobe, is also a characteristic feature of AD [240]. 
Furthermore, inflammation and oxidative stress have been implicated in the 
pathogenesis of AD, with activation of immune cells and release of 
inflammatory chemicals contributing to neuronal damage [242]. 


Risk Factors and Contributing Factors 


While the exact causes of AD are still unknown, several risk factors and 
contributing factors have been identified. Genetic factors, such as the APOE 
€4 allele and mutations in APP, PSEN1, and PSEN2 genes, increase the risk of 
developing AD [241]. Environmental and lifestyle factors, including age, 
cardiovascular health, head injuries, physical inactivity, smoking, and poor 
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diet, also contribute to the risk of AD. Social engagement and cognitive reserve 
have also been shown to play a role in AD risk, with low social engagement 
and loneliness increasing risk and higher cognitive reserve delaying onset 
[243]. 


Current Understanding and Future Directions 


Despite significant advances in our understanding of AD, much remains to be 
discovered. Current research is focused on developing effective treatments and 
prevention strategies, including immunotherapies targeting amyloid-beta and 
tau and lifestyle interventions aimed at reducing risk factors. A comprehensive 
understanding of the complex interplay between genetic, environmental, and 
lifestyle factors will be crucial in developing effective prevention and 
treatment strategies for AD. 
Infographic Summary: 
e Symptoms: Memory loss, confusion, difficulty with language, 
disorientation, mood swings, and behavioral changes. 
e Stages: Mild (early-stage), moderate (middle-stage), and severe (late- 
stage). 
e Statistics: It affects approximately 50 million people worldwide; the 
incidence is expected to triple by 2050. 
e Key Biomarkers: Amyloid-beta plaques, tau tangles, and brain 
atrophy. 


Parkinson's Disease 


A Progressive Movement Disorder 


Parkinson's disease (PD) is a chronic and progressive neurodegenerative 
disorder that primarily affects motor function. It is the second most common 
neurodegenerative disease after Alzheimer's, characterized by symptoms such 
as tremors, rigidity, bradykinesia (slowness of movement), and postural 
instability [244]. Non-motor symptoms, including cognitive impairment, 
mood disorders, sleep disturbances, and autonomic dysfunction, are also 
prevalent [245]. The progression of PD varies, but it generally leads to a 
significant decline in quality of life and functional independence. 
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Figure 12.1: (A) loss of dopaminergic (DA) neurons in the substantia nigra (SN), (B) Healthy 
vs PD Patient presynaptic terminal, (C) resulting motor symptoms [246] 


Pathophysiological Mechanisms 


The hallmark of PD is the degeneration of dopaminergic neurons, shown in 
Fig. 12.1 [246], in the substantia nigra, a region of the brain that plays a crucial 
role in regulating movement. The loss of these neurons leads to a significant 
decrease in dopamine levels, disrupting normal communication between the 
brain and muscles and resulting in the motor symptoms characteristic of PD 
[247]. Another key feature of PD is the presence of Lewy bodies—abnormal 
aggregates of alpha-synuclein protein within neurons. These aggregates are 
toxic to cells and contribute to neurodegeneration [248]. Recent research also 
suggests a potential gut-brain axis involvement, where misfolded alpha- 
synuclein may originate in the gastrointestinal tract and travel to the brain via 
the vagus nerve, implicating the gut in PD pathogenesis [249]. 


Risk Factors and Contributing Factors 


PD's etiology (cause) involves genetic and environmental factors. Genetic 
mutations in genes such as LRRK2, PARK2, and SNCA increase the risk of 
developing PD, although these account for only a small percentage of cases 
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[244]. Environmental factors, including exposure to pesticides, heavy metals, 
and solvents, have been associated with a higher risk of PD [250]. 
Additionally, age is the most significant risk factor, with most cases occurring 
in individuals over 60. There is also evidence that rural living and farming 
occupations, likely due to increased exposure to agricultural chemicals, are 
associated with higher PD prevalence [250]. 


Current Understanding and Future Directions 


While there is currently no cure for PD, various treatments aim to manage 
symptoms and improve quality of life. Medications such as levodopa (L-Dopa) 
and dopamine agonists are commonly used to replace or mimic dopamine, 
alleviating motor symptoms [244]. Deep brain stimulation (DBS) is a surgical 
intervention that can relieve some patients. Despite hope in_ these 
symptomatic-based treatments, some significant side effects accompany them. 
Some are loss of efficacy over time, hallucinations, involuntary and 
compulsive behaviors, and involuntary movements. Future research is focused 
on understanding the mechanisms of alpha-synuclein aggregation, exploring 
neuroprotective strategies, and investigating the potential role of the gut 
microbiome in PD. Rather than just addressing symptoms, developing 
treatments that modify the disease course remains a critical goal. 

Infographic Summary: 

e Symptoms: Tremors, muscle rigidity, bradykinesia, postural 
instability, cognitive impairment, mood disorders, and_ sleep 
disturbances. 

e Stages: Early-stage, mid-stage, and advanced-stage. 

e Statistics: It affects over 10 million people worldwide and is more 
common in men than women. 

e Key Biomarkers: Dopaminergic neuron loss, Lewy bodies, and 
reduced dopamine levels. 
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Autism Spectrum Disorder (ASD) 


A Complex Developmental Condition 


Autism Spectrum Disorder (ASD) is a developmental disorder that affects 
communication, behavior, and social interaction. The term "spectrum" reflects 
the wide range of symptoms and severity observed among individuals with 
ASD [251]. Core features include difficulties in social communication, 
restricted interests, and repetitive behaviors. ASD can present with various 
cognitive abilities, from intellectual disability to high functioning. Still, there 
is a range of symptoms associated with ASD that may be different from male 
to female [251]. 


Pathophysiological Mechanisms 


ASD is associated with atypical brain development, including differences in 
structure and connectivity. These neurodevelopmental anomalies may affect 
neural circuits involved in social communication and behavior [252]. Genetic 
factors play a significant role in ASD, with many implicated genes affecting 
synaptic function and neural development [253]. Environmental factors, such 
as prenatal exposures to certain drugs, infections, or toxins, may also 
contribute to the risk [254]. Epigenetic mechanisms, which involve changes in 
gene expression without altering the DNA sequence, are thought to play a role 
in how genetic and environmental factors influence brain development in ASD 
[255]. 


Risk Factors and Contributing Factors 


ASD has a strong genetic component, with heritability estimates ranging from 
50% to 90% [253]. Several genetic syndromes, such as Fragile X syndrome 
and Rett syndrome, are associated with a higher risk of ASD [251]. 
Environmental factors during pregnancy, including maternal infections and 
exposure to certain chemicals, have been linked to ASD. Advanced parental 
age, particularly in fathers, has also increased risk [254]. The disorder is more 
commonly diagnosed in males, with a male-to-female ratio of approximately 
4:1 [251]. 
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Current Understanding and Future Directions 


ASD diagnosis typically involves behavioral assessments and developmental 
history. Early intervention, including behavioral therapies and educational 
support, can significantly improve outcomes. Current research focuses on 
understanding the underlying genetic and neurobiological mechanisms of 
ASD, developing biomarkers for earlier and more accurate diagnosis, and 
exploring new therapeutic interventions. Advances in technology, such as AI 
for behavioral analysis and VR for social skills training, promise to enhance 
the lives of individuals with ASD. 
Infographic Summary: 
e Symptoms: Challenges in social interaction, communication 
difficulties, repetitive behaviors, and restricted interests. 
e Diagnostic Criteria‘ Based on behavioral observations and 
developmental history. 
e Statistics: It affects 1 in 54 children in the United States; it is more 
common in boys than girls. 
e Key Biomarkers: Abnormal brain connectivity, structural differences 
in the brain, and genetic mutations. 


Multiple Sclerosis (MS) 


An Autoimmune and Neurodegenerative Disorder 


Multiple Sclerosis (MS) is a chronic autoimmune disorder that affects the 
central nervous system (CNS), leading to demyelination and 
neurodegeneration. It has various symptoms, including motor and sensory 
impairments, visual disturbances, and cognitive changes. The course of the 
disease can vary, with some individuals experiencing relapsing-remitting 
episodes and others progressing to a more severe form known as secondary 
progressive MS [256]. 


Pathophysiological Mechanisms 


MS involves an immune-mediated attack on the myelin sheath (Chapter 1), the 
protective covering around nerve fibers in the CNS. This demyelination 
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disrupts the transmission of electrical impulses along the nerves, leading to 
symptoms such as muscle weakness, spasticity, and sensory disturbances 
[257]. In addition to demyelination, MS can cause axonal damage and 
neuronal loss, contributing to permanent neurological deficits [258]. The 
disease is associated with chronic inflammation, involving T and B cells that 
cross the blood-brain barrier and attack CNS components [257]. 
Neurodegeneration, resulting from chronic inflammation and immune- 
mediated damage, is a significant aspect of MS pathology [258]. 


Risk Factors and Contributing Factors 


Genetic factors play a role in MS, with specific genetic variations, particularly 
in the HLA-DRB1*15:01 allele, associated with increased susceptibility [259]. 
Environmental factors also contribute to MS risk, including viral infections 
(such as Epstein-Barr virus), vitamin D deficiency, and smoking. Geographic 
factors are notable, with higher MS prevalence observed in regions farther 
from the equator, suggesting a link to sunlight exposure and vitamin D levels 
[260]. 


Current Understanding and Future Directions 


MS is typically diagnosed through clinical evaluation, MRI, and cerebrospinal 
fluid analysis. Disease-modifying therapies (DMTs) are available to reduce the 
frequency and severity of relapses and slow disease progression. Current 
research focuses on understanding the mechanisms of immune dysregulation 
and neurodegeneration in MS, developing more effective DMTs, and exploring 
strategies for promoting remyelination and neuroprotection. Advances in 
imaging and biomarker discovery are also key focus areas, aiming to improve 
early diagnosis and treatment monitoring. 
Infographic Summary: 

e Symptoms: Fatigue, motor and sensory impairments, vision problems, 

cognitive dysfunction, and balance issues. 


e Types: Relapsing-remitting MS (RRMS), secondary progressive MS 
(SPMS), primary progressive MS (PPMS). 
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e Statistics: It affects approximately 2.8 million people worldwide, more 
common in women than men. 

e Key Biomarkers: Demyelination, oligoclonal bands in cerebrospinal 
fluid (CSF), and MRI-detected lesions. 


Amyotrophic Lateral Sclerosis (ALS) 


A Degenerative Motor Neuron Disease 


Amyotrophic Lateral Sclerosis (ALS), also known as Lou Gehrig's disease, is 
a progressive neurodegenerative disorder that primarily affects motor neurons. 
These neurons control voluntary muscles, and degeneration leads to muscle 
weakness, atrophy, and, eventually, paralysis. The disease progresses rapidly, 
with most patients succumbing to respiratory failure within 3-5 years of 
diagnosis [261]. 


Pathophysiological Mechanisms 


ALS involves the degeneration of both upper motor neurons (in the brain) and 
lower motor neurons (in the brainstem and spinal cord). This degeneration 
disrupts the transmission of signals from the brain to the muscles, resulting in 
the progressive loss of motor function [262]. The presence of abnormal protein 
aggregates, such as TDP-43, SOD1, and FUS, is a common pathological 
feature in ALS, contributing to neuronal toxicity and cell death [263]. Elevated 
levels of glutamate, an excitatory neurotransmitter, have been implicated in 
ALS, causing excitotoxicity and further neuronal damage. Oxidative stress and 
mitochondrial dysfunction are also involved, contributing to the pathogenesis 
and progression of the disease. The stages of ALS can be seen in Table 12.1 
[264]. 


Risk Factors and Contributing Factors 


Approximately 10% of ALS cases are familial, meaning they are inherited. 
Several genetic mutations have been identified, including those in the SOD1, 
C9orf72, TARDBP, and FUS genes, with C9orf72 being the most common 
genetic cause [265]. Environmental factors may also contribute to ALS risk, 
including exposure to toxins, heavy metals, pesticides, and physical trauma. 
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Smoking and a history of intense physical activity have been associated with 
increased risk. The disease is more commonly diagnosed in individuals 
between 40 and 70 and is slightly more prevalent in men [261]. 


Table 12.1: Stages of ALS and its associated Symptoms and Physical Effects [259] 


Stage Symptoms Physical Effects 


Early Muscle weakness, | Fatigue, poor balance, slurred 
fasciculations, and atrophy | words, tripping, and weak grip 
are often limited to one 
body region. 


Middle | The symptoms seen in the | Muscle contractures, weakness in 
early stages are more | breathing and swallowing causing 
widespread and affect more | difficulty eating, drinking, and 
than one body region. | breathing 

Muscles become paralyzed, 


and fasciculations 
continue. 

Late Most voluntary muscles are | Very limited mobility, poor 
paralyzed, and the | respiration causing fatigue, and 
breathing muscles are very | increased susceptibility to 
weak. pneumonia. Loss of speech and 


limited eating/drinking via mouth 


Current Understanding and Future Directions 


There is currently no cure for ALS, and treatment focuses on managing 
symptoms and improving quality of life. Riluzole and edaravone are the only 
FDA-approved drugs that modestly extend survival or slow disease 
progression. Non-invasive ventilation (NIV) and other supportive therapies 
can help manage respiratory complications and maintain communication 
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abilities. Current research focuses on understanding the genetic and molecular 
mechanisms underlying ALS, developing neuroprotective and 
neurorestorative therapies, and exploring potential treatments such as gene and 
stem cell therapy. 


Dementia 


A Spectrum of Neurodegenerative Disorders 


Dementia is an umbrella term used to describe a range of neurodegenerative 
conditions characterized by progressive cognitive decline and impairment in 
daily functioning. The most common type of dementia is Alzheimer's disease, 
followed by vascular dementia, Lewy body dementia, and frontotemporal 
dementia [241]. Dementia affects memory, thinking, behavior, and the ability 
to perform everyday activities. 


Pathophysiological Mechanisms 
Alzheimer's Disease Pathology 


Alzheimer's disease is characterized by the accumulation of amyloid-beta 
plaques and tau protein tangles in the brain, which disrupt neuronal 
communication and lead to cell death and brain atrophy [265]. 
Vascular Dementia 

This type of dementia results from reduced blood flow to the brain, often due 
to strokes or other vascular issues, leading to ischemic damage and cell death 
[266]. 

Lewy Body Dementia 

In this condition, abnormal clumps of alpha-synuclein protein, known as Lewy 
bodies, are present in the brain. These contribute to cognitive decline, motor 
symptoms like Parkinson's disease, and vivid hallucinations[267]. 
Frontotemporal Dementia 

This form of dementia involves degeneration of the frontal and temporal lobes, 
affecting personality, behavior, and language abilities [268]. 
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Risk Factors and Contributing Factors 


Genetic factors contribute to the risk of developing dementia, with mutations 
in specific genes associated with early-onset forms of Alzheimer's disease and 
other dementias. Cardiovascular risk factors, such as hypertension, diabetes, 
and high cholesterol, are significant contributors, particularly for vascular 
dementia [266]. Lifestyle factors, including diet, physical activity, and 
cognitive engagement, also play crucial roles in the risk of developing 
dementia [241]. Age is the most significant risk factor, with the prevalence of 
dementia increasing significantly in individuals over the age of 65. 


Current Understanding and Future Directions 


Diagnosis of dementia involves a combination of medical history, cognitive 
testing, and imaging techniques. There is currently no cure for dementia, and 
treatments focus on managing symptoms and slowing disease progression. 
Research is ongoing to understand the underlying mechanisms of various types 
of dementia, develop biomarkers for early diagnosis, and explore potential 
disease-modifying therapies. Advances in personalized medicine and 
technology, such as cognitive training programs and Al-powered diagnostic 
tools, offer hope for better management and improved outcomes for 
individuals with dementia. 
Infographic Summary: 
e Symptoms: Memory loss, confusion, language difficulties, impaired 
judgment, and changes in behavior and personality. 
e Types: Alzheimer's disease, vascular dementia, Lewy body dementia, 
frontotemporal dementia. 
e Statistics: It affects approximately 50 million people worldwide and is 
expected to increase as the population ages. 
e Key Biomarkers: Varies by type; includes amyloid-beta plaques, tau 
tangles, Lewy bodies, and ischemic lesions. 
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Huntington's Disease 


A Hereditary Neurodegenerative Disorder 


Huntington's disease (HD) is a progressive neurodegenerative disorder caused 
by a genetic mutation. It is characterized by motor, cognitive, and psychiatric 
symptoms. The disease typically manifests in mid-adulthood and 
progressively worsens over time, leading to significant disability and early 
death [269]. The prevalence of HD is estimated to be about 5-10 cases per 
100,000 individuals of European descent [270]. 


Pathophysiological Mechanisms 


HD is caused by an expanded CAG repeat in the HTT gene, which encodes 
the huntingtin protein. Individuals with more than 36-40 CAG repeats in the 
HTT gene will invariably develop the disease [38]. This expansion produces 
an abnormally long polyglutamine tract in the huntingtin protein, causing it to 
misfold and form toxic aggregates in neurons [271]. These aggregates disrupt 
normal cellular processes, including transcriptional regulation, mitochondrial 
function, and protein degradation pathways, leading to neuronal dysfunction 
and death [271]. The most affected brain regions are the striatum and cerebral 
cortex, which are crucial for motor control and cognitive functions [272]. 


Risk Factors and Contributing Factors 


HD is inherited in an autosomal dominant manner, meaning an individual only 
needs one copy of the mutated gene from one parent to develop the disease. 
The number of CAG repeats in the HTT gene correlates with the age of onset 
and severity of symptoms; a higher number of repeats is associated with an 
earlier onset and more severe disease course [273]. Unlike many other 
neurodegenerative disorders, environmental factors do not significantly 
influence the onset or progression of HD, as the disease's course is primarily 
determined by genetic factors [274]. 


Current Understanding and Future Directions 


There is currently no cure for HD, and treatment primarily focuses on 
managing symptoms. Medications such as tetrabenazine can help control 


491 


chorea (involuntary movements), while antipsychotics and antidepressants are 
used to manage psychiatric symptoms. Ongoing research is exploring potential 
disease-modifying therapies, including gene silencing techniques, such as 
antisense oligonucleotides and RNA interference, to reduce the production of 
the mutant huntingtin protein. Additionally, studies are investigating 
neuroprotective strategies and the use of stem cell therapies to replace lost 
neurons and restore brain function. 
Infographic Summary: 
e Symptoms: Involuntary movements (chorea), impaired coordination, 
cognitive decline, mood swings, and psychiatric symptoms. 
e Stages: Early-stage, mid-stage, and late-stage. 
e Statistics: It affects approximately 5-10 per 100,000 people 
worldwide. 
e Key Biomarkers: Genetic testing for HTT gene mutation, MRI 
showing brain atrophy. 


Impact on Society 


Neurological disorders have a profound impact on individuals, families, and 
society. The effects of these disorders extend far beyond the patients 
themselves, affecting employers, caregivers, physicians, insurance companies, 
and regulatory bodies. For instance, patients with neurological disorders often 
experience a reduced quality of life due to cognitive decline, mobility issues, 
and emotional distress. This can lead to a loss of independence and autonomy, 
increasing the risk of comorbidities and secondary health conditions [275]. 
Furthermore, the emotional and psychological burden of neurological 
disorders should not be underestimated, with anxiety, depression, and stigma 
being common comorbidities. 

Caregivers, often family members or friends, also bear a significant burden. 
The emotional strain and stress of caring for a loved one can be overwhelming, 
leading to physical demands such as sleep disturbances and fatigue. 
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Figure 12.2: DALYs by age group and categorized by disorders [278] 


Additionally, caregivers may experience a financial burden due to reduced 
income, increased expenses, and lost productivity. Social isolation and reduced 
social connections are also common consequences of caregiving, highlighting 
this critical group's need for support and resources [276]. 

Employers are also impacted by neurological disorders, with reduced 
productivity and absenteeism being common consequences of employees 
caring for loved ones. Increased healthcare costs and insurance premiums are 
additional burdens, with a potential loss of skilled and experienced employees 
due to caregiving responsibilities [277]. As such, employers must consider 
accommodations and support for employees with neurological disorders, 
ensuring a healthy and productive work environment. 

Physicians, too, face unique challenges in caring for patients with neurological 
disorders. The emotional toll of caring for patients with debilitating and 
incurable conditions should not be underestimated, with complexities of 
diagnosis, treatment, and management adding to the burden [40]. Staying 
current with the latest research, treatments, and guidelines is essential while 
balancing patient care with administrative and regulatory responsibilities. This 
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is especially critical in rural communities, low-income areas, and low-income 
countries where access to specialized care is not readily available. 

Insurance companies face increased costs due to healthcare services, long-term 
care, and prescription medications, and they face challenges in assessing and 
managing risk for neurological disorders. Specialized programs and services 
for patients with neurological disorders are essential, and if not managed 
effectively, they can potentially increase premiums and reduce coverage 
[276]. 

Regulatory bodies are critical in developing and implementing policies and 
guidelines for neurological disorders, ensuring patient and caregiver access to 
care, services, and support [275]. Monitoring and addressing healthcare 
disparities and inequities is essential, balancing individual needs with societal 
and economic considerations a delicate task. 

The economic impact of neurological disorders is substantial, with estimated 
annual costs of $305 billion for Alzheimer's disease [241], $52 billion for 
Parkinson's disease (Parkinson's Foundation, 2022, $15 billion for epilepsy 
[277], and $10 billion for multiple sclerosis [276]. These figures highlight the 
need for continued research, support, and resources to address the complex 
needs of patients, caregivers, and society. The DALYs (Disability-adjusted life 
years) refer to years of healthy life lost due to premature death and disability 
(Fig. 12.2 [278]) and are a good comparison to the burden of disease. 


Technological Interventions in Neurological 
Disorders 


The Role of Technology 


Technological advancements have significantly impacted the field of 
neuroengineering, providing innovative tools for diagnosing, treating, and 
managing neurological disorders. These advancements span various 
technologies, including advanced imaging techniques, wearable devices, 
artificial intelligence (AI), and machine learning (ML). Integrating these 
technologies into clinical practice can revolutionize neurological care, offering 
more precise diagnostics, personalized treatments, and improved patient 
outcomes. 
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Advanced Imaging Techniques 


Advanced imaging techniques, such as magnetic resonance imaging (MRI) 
(Chapter 5), positron emission tomography (PET), and functional MRI (fMRI) 
(Chapter 5), have enhanced the ability to diagnose and monitor neurological 
disorders. These modalities provide detailed insights into brain structure and 
function, facilitating early detection of abnormalities and precise tracking of 
disease progression. For instance, MRI and PET scans are commonly used in 
diagnosing and monitoring Alzheimer's disease, allowing visualization of 
amyloid plaques and tau tangles—key biomarkers of the disease [241]. These 
imaging technologies are also vital in assessing structural and functional 
changes in Parkinson's disease, multiple sclerosis, and _ other 
neurodegenerative conditions [240]. 


Wearable Devices 


Wearable devices have gained prominence in managing neurological 
disorders, continuously monitoring physiological parameters such as heart 
rate, movement, and sleep patterns. These devices provide valuable data for 
patients and clinicians, enabling more accurate disease management. In 
Parkinson's disease, wearable sensors can track motor symptoms like tremors 
and bradykinesia, aiding treatment optimization [279]. Similarly, wearable 
devices monitor and manage epilepsy by detecting and recording seizure 
activity, allowing timely interventions. 


Artificial Intelligence and Machine Learning 


AI and ML technologies, Discussed in Chapter 3, are increasingly employed 
in diagnosing and treating neurological disorders. These technologies can 
analyze large datasets to identify patterns and predict outcomes, potentially 
leading to earlier diagnoses and more personalized treatment plans. For 
example, AI algorithms have been developed to predict the onset of 
Alzheimer's disease using imaging and genetic data [83]. Additionally, ML 
models optimize deep brain stimulation (DBS) parameters in Parkinson's 
disease, enhancing the efficacy of this treatment [280]. The application of AI 
extends to various disorders, including multiple sclerosis and dementia, where 
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it aids in diagnosing, monitoring disease progression, and developing 
individualized treatment strategies. 
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Figure 12.3: Telemedicine Growth During Covid-19 between 2019 and 2021 [281] 


Telemedicine 


The adoption of telemedicine has been accelerated by the COVID-19 
pandemic [281], which has proven to be a valuable tool for managing 
neurological disorders (Fig. 12.3 [281]). Telemedicine platforms enable 
remote consultations with neurologists, reducing the need for travel and 
improving access to care, particularly for patients with mobility issues or those 
in remote areas. Studies have demonstrated the effectiveness of telemedicine 
in managing conditions such as epilepsy, multiple sclerosis, and headache 
disorders, providing continuous care and monitoring [282]. 


Technological Interventions by Disorder 


Alzheimer's Disease 


Current Technology 
1. Brain Imaging Techniques: MRI and PET scans are integral in 
detecting brain changes associated with Alzheimer's disease. These 
technologies help identify brain atrophy and the presence of amyloid- 
beta plaques and tau tangles [241]. 
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2. Wearable Devices: Devices that monitor cognitive function and daily 
activities can detect early signs of cognitive decline, providing 
continuous patient monitoring [283]. 

3. AlI-Based Diagnostic Tools: AI and ML algorithms analyze medical 
imaging and genetic data to improve diagnostic accuracy, facilitating 
early and precise diagnoses [284]. 

Future Directions 

Research is focused on developing neuroprotective drugs to slow or halt 
disease progression. Advances in biomarker identification could enable earlier 
diagnoses, while personalized medicine approaches tailored to individual 
genetic profiles are being explored to enhance treatment effectiveness. 
Parkinson's Disease 

Current Technology 

1. Deep Brain Stimulation (DBS): DBS involves implanting electrodes 
in specific brain areas to regulate abnormal brain activity, significantly 
improving motor symptoms in Parkinson's patients [280]. 

2. Wearable Sensors: These devices monitor motor symptoms and 
provide real-time data, assisting clinicians in adjusting treatment plans 
and medications more accurately [285]. 

3. Mobile Apps: Apps help manage medication schedules, track 
symptoms, and provide exercise routines, supporting patients in 
maintaining structured care routines. 


Future Directions 


Future research in Parkinson's focuses on gene therapy and stem cell therapy. 
Gene therapy aims to correct genetic mutations, while stem cell therapy seeks 
to replace lost dopaminergic neurons. Neurorestorative therapies are also being 
explored to repair damaged neural circuits. 
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Autism Spectrum Disorder (ASD) 


Current Technology 

1. Assistive Communication Devices: Augmentative and alternative 
communication (AAC) devices, including speech-generating devices 
and apps, facilitate communication for individuals with ASD [286]. 

2. Virtual Reality (VR): VR technology creates immersive environments 
for practicing social interactions and developing social skills [226]. 

3. AJ-Based Behavioral Analysis: AI algorithms analyze behavioral data, 
identifying patterns and providing insights for more effective behavior 
intervention plans [287]. 


Future Directions 

Ongoing research aims to develop advanced genetic testing for early detection 
and personalized intervention programs tailored to each individual's genetic 
profile. AI continues to enhance the accuracy and effectiveness of behavioral 
interventions. 


Multiple Sclerosis (MS) 


Current Technology 

1. MRI for Disease Monitoring: MRI is the gold standard for diagnosing 
and monitoring MS, providing detailed images of brain and spinal 
cord lesions [288]. 

2. Disease-Modifying Therapies (DMTs): DMTs reduce relapse 
frequency and disease progression, targeting the immune system to 
prevent myelin damage [289]. 

3. Telemedicine: Telemedicine facilitates remote consultations, 
improves access to care, and enables regular monitoring [290]. 


Future Directions 


Future research aims to develop advanced immunotherapies and 
neuroprotective agents to prevent or repair myelin damage. Remyelination 
strategies are also being explored to restore function. 
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Dementia 


Current Technology 
1. Cognitive Training Programs: Computer-based programs improve 
memory, attention, and executive function in individuals with 
dementia [291]. 
2. GPS Tracking Devices: These devices monitor the location of 
individuals who may wander, enhancing safety [292]. 
3. AI and Robotics: Al-powered virtual assistants and robotic 
companions offer cognitive and social support, improving quality of 
life [293]. 
Future Directions 
Research is focused on developing more advanced cognitive training programs 
leveraging VR and gamification. AI is being explored to predict disease 
progression and tailor interventions. 


Huntington's Disease 


Current Technology 

1. Genetic Testing: Genetic testing is crucial for diagnosing Huntington's 
disease and assessing risk in individuals with a family history [269]. 

2. Symptomatic Treatments: Medications such as tetrabenazine manage 
motor symptoms, while antipsychotics address psychiatric issues 
[294]. 

3. Telemedicine: Telemedicine platforms provide access to specialized 
care, support regular monitoring, and reduce the need for travel. 


Future Directions 


Research focuses on developing disease-modifying therapies targeting the 
genetic mutation responsible for Huntington's. Gene editing technologies, such 
as CRISPR, hold promise for correcting the mutation and potentially halting 
disease progression. 
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Future Directions 


In the coming years, advancements in neuroengineering are expected to 
significantly improve our understanding, diagnosis, and treatment of 
neurological disorders. One exciting area of development is neuroprotective 
and neurorestorative therapies. These treatments aim to protect neurons from 
damage and potentially restore lost functions in conditions like Alzheimer's 
and Parkinson's disease. Imagine being able to slow down or even stop the 
progression of these diseases, giving people more time with their loved ones 
and a better quality of life. 

Another promising direction is the use of biomarkers for early diagnosis and 
monitoring. Biomarkers are measurable indicators found in_ blood, 
cerebrospinal fluid, or even through imaging. They can provide valuable 
information about what's happening in the brain, sometimes before symptoms 
start. Early detection can lead to earlier treatments, which might slow down or 
alter the course of the disease, offering a critical intervention window. 

The integration of genomics and personalized medicine into treatment plans is 
another game-changer. By understanding a patient's genetic makeup, doctors 
can tailor treatments to suit their needs, potentially leading to better outcomes. 
This is especially relevant for diseases with a genetic component, like 
Huntington's disease. Personalized medicine is about more than just using the 
right drugs—it's about finding the right treatments at the right time for 
everyone. 

Brain-computer interfaces (BCIs) and neural prosthetics also push the 
boundaries of what's possible. BCIs create a direct communication pathway 
between the brain and external devices, which can help people with conditions 
like ALS regain some control over their environment. Neural prosthetics, like 
cochlear implants, are already assisting people to restore lost senses, and new 
developments promise even more exciting applications. 


Chapter 12: Summary 


Neurological disorders are challenging for everyone involved, from the 
patients and their families to the healthcare professionals providing care. The 
increasing prevalence of these conditions and the growing older population 
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make it clear that we must continue to push forward with research and 
development in this field to relieve the burden on healthcare systems and 
society. 

For those of you in neuroengineering, this is an incredibly exciting time. 
There's so much potential to make a real difference in people's lives. Whether 
you're interested in developing new diagnostic tools, working on cutting-edge 
therapies, or improving existing technologies, there's a place for your skills 
and creativity. 

Researchers and healthcare professionals need to stay updated on the latest 
advancements and be ready to incorporate new tools and treatments into their 
practice. Policymakers and funding organizations also have a crucial role. 
They must support this research and ensure new technologies can be safely 
and quickly brought to market. 

Finally, raising public awareness about neurological disorders is essential. The 
more people understand these conditions, the better support systems we can 
create for those affected. This also helps reduce the stigma around neurological 
disorders, making it easier for people to seek help. 

In summary, neuroengineering offers countless opportunities to innovate and 
improve the lives of those with neurological disorders. By embracing new 
technologies and working together across disciplines, we can make significant 
strides in understanding, treating, and preventing these conditions. So, whether 
you're just starting your studies or already deep into research, now is the time 
to get involved and make an impact. 
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Chapter 12: Learning activities 


Learning Activity 12.1 


Activity: Identifying Neurological Disorders 
in Case Studies 


Objective 
Students will analyze case studies to identify neurological disorders and 
respond to questions to deepen their understanding of neurocognitive 
disorders. 


Materials 


e Internet access 

e Computers or tablets 

e Printed or digital copies of the case study: Case Studies: 
Neurocognitive Disorders 


e Note-taking materials (paper, pens, or digital tools) 
Time 
30-40 minutes 


Instructions 


Introduction (5 minutes): 
Introduce the activity and explain its goal: to identify neurological disorders 
through analyzing case studies. 


1. Briefly overview the importance of understanding neurocognitive 
disorders in neuroengineering and psychology. 


Case Study Analysis (20 minutes): 

3. Distribute printed or digital copies of the case study to each student or group. 
4. Instruct students to read the case study carefully and take notes on key 
symptoms, behaviors, and any relevant background information. 


502 


Question Response (10 minutes): 


5. Ask students to respond individually or in groups to the questions in the case 
study. Ensure they reference specific details from the case to support their 
answers. 


Group Discussion (5 minutes): 


6. Facilitate a class discussion where students can share their findings and 
compare their answers. Encourage them to explain their reasoning and listen 
to different perspectives. 


Conclusion (5 minutes): 

7. Summarize the key points from the discussion, highlighting the identified 
neurological disorders and the symptoms that led to these conclusions. 8. 
Emphasize the relevance of these skills in real-world applications, such as 
diagnosing and treating neurocognitive disorders. 


Learning Activity 12.2 


Activity: Discovery Challenge - Identifying 
Neuro Diseases from NIH FOAs 


Objective 


Students will explore the NIH website to quickly identify and collect a list of 
neuro diseases mentioned in the published Funding Opportunity 
Announcements (FOAs). 

Materials 


e Internet access 
e Computers or tablets 
e Note-taking materials (paper, pens, or digital tools) 


Time 


10 minutes 
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Instructions 


Introduction (2 minutes): 


1. Introduce the activity and explain its goal: to discover how many 
neuro diseases are mentioned in the NIH FOAs within a 5-minute 
time frame. 

2. Briefly explain what FOAs are and their relevance in research 
funding. 


Discovery Task (5 minutes): 

3. Instruct students to visit the NIH website and navigate to the section with 
published FOAs. 

4. Ask students to find and list as many neuro diseases as possible within the 
5-minute time limit. 


Group Sharing (3 minutes): 
5. Have students share the number and names of the neuro diseases they found. 


6. Compile a class list on the whiteboard or digital platform, noting any 
overlaps and unique findings. 


@ 


rr 
PAs 
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Chapter 12: Lab introduction 


In this series of lab exercises, you will explore advanced concepts in 
neuroscience through practical applications. These labs provide 
hands-on experience simulating neural activity and 
neurostimulation, deepening your understanding of how 
neurological disorders and interventions impact brain function. 

In the first lab, you will expand your knowledge of the Hodgkin- 
Huxley (HH) model by simulating neural activity differences 
between a healthy brain and a brain affected by Parkinson's disease. 
You will create a graphical user interface (GUI) to input various 
parameters and observe their impact on the membrane potential of 


neurons. This exercise will help you understand how changes in the 


HH model parameters can represent the effects of neurological 
diseases. 

The second lab combines concepts from neurostimulation, brain- 
computer interfaces (BCIs), and neurological disorders. You will 
simulate neurostimulation's impact on conditions like Parkinson's 
disease and depression using coil designs. Additionally, you will 
conduct statistical analysis to compare the effects of 
neurostimulation on these disorders, providing insights into how 
such interventions can modulate neural activity to alleviate 
symptoms. 
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Chapter 12: Lab Example 1 


Overview 


In this lab example, we will continue with our Neuron 
Example in Chater 2 and focus on modifying parameters for 
Parkinson’s Disease. We will focus on expanding some understanding of the 


HH model and how diseases in this chapter influence the model. 

In this lab example, we will simulate the differences in neural activity between 
a healthy brain and a brain affected by Parkinson's disease using the Hodgkin- 
Huxley (HH) model. We will create a graphical user interface (GUI) that 
allows us to input various parameters and observe their impact on the 
membrane potential of neurons. 


Requirements 


1. Python (tested with 3.8 and later) 
2. NEURON (neuron.yale.edu) 

3. Matplotlib 

4. Tkinter 


Steps 

The first thing you need to do is go to the GitHub repository for this book. You 
can find it using this link: 

https://github.com/bddupre92/Neurobook BME UND 

After you have opened the GitHub, you need to navigate to Chapter 12, Lab 


Example 1. In this lab example, you will find the code for the simulation. 
You will want to copy and paste the code into a script named 
PDNeuronGUL py. 


Script Structure 


Define Constants and Parameters: 


1. amp_value: Amplitude of the current. 
2. duration_value: Duration of the current. 
3. num _ steps value: Number of steps in the simulation. 
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dend nseg value: Number of segments in the dendrite. 

cm_value: Membrane capacitance. 

Ra_value: Axial resistance. 

gbar_hh_value: Maximum conductance for sodium, potassium, and 
leak channels. 

v_init_value: Initial membrane potential. 


The GUI input will pop-up with pre-populated values for everything. Run the 


initial example before modifying parameters to assess if this is a good starting 
point for the HH Model. 


Baseline Neural Activity Simulation 


Simulate baseline neural activity with the defined parameters. 


Post-Stimulation Neural Activity Simulation 


Simulate neural activity after modifying the parameters to represent 


Parkinson's disease. 


Visualization 


Plot the membrane potential for both the healthy and Parkinson's brain models. 
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G HH Model Simulation o x 


Amplitude of Current (nA): 1 
Duration of Current (ms): 5 | 
Number of Steps: \5 
Number of Segments (nseg): 10 
Membrane Capacitance (cm): 1 
Axial Resistance (Ra): 100 
Na Conductance (gbar_na): 0.12 
K Conductance (gbar_k): 0.036 
Leak Conductance (gbar_l): 0.0003 
Resting Membrane Potential (v_init): es CC 


Run Simulation 


Figure 12.4: Model Parameters 


Explanation of the Figures 


Duration vs. Membrane Potential 
e Fig. 12.5 shows how the duration of the injected current affects the 
membrane potential. 
e By varying the duration, you can observe the impact on the action 
potential duration and how prolonged stimulation influences the 
neuron's behavior. 
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Duration vs. Membrane Potential 
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Figure 12.5: Duration vs. Membrane Potential 


Current vs. Membrane Potential 


e Fig. 12.6 shows how different amplitudes of the injected current affect 
the membrane potential for healthy and Parkinson's models. 


Current vs. Membrane Potential 
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Figure 12.6: Current vs. Membrane Potential 
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e By varying the current amplitude, you can observe the threshold 
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Figure 12.7: Membrane Capacitance (cm) Sensitivity 


Membrane Potential Comparison 
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Figure 12.8: Membrane Potential Comparison 
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behavior and how the neurons respond differently in healthy versus 
Parkinson's conditions. 


Explanation of the Parameter Sensitivity Plots 


Membrane Capacitance (cm) Sensitivity 
e Fig. 12.7 shows how changes in membrane capacitance affect the 
membrane potential. 
e Higher capacitance can make the neuron less responsive, while 
lower capacitance makes it more responsive. 


Axial Resistance (Ra) Sensitivity 
e Illustrates the impact of axial resistance on the membrane 
potential. 
e Higher resistance slows down signal propagation, while lower 
resistance speeds it up. 


Sodium Conductance (gbar_na) Sensitivity 
e Displays how changes in sodium conductance influence the action 
potential. 
e Higher conductance increases excitability, making it easier to 
trigger action potentials. 


Potassium Conductance (gbar_k) Sensitivity 
e Demonstrates the effect of potassium conductance on 
repolarization. 
e Higher conductance leads to quicker repolarization, shortening the 
action potential duration. 


Leak Conductance (gbar_l) Sensitivity 
e Shows how variations in leak conductance affect the membrane 
potential. 
e Higher leak conductance stabilizes the resting membrane potential 


but reduces overall excitability. 
These sensitivity plots help understand the model's robustness and the 
significance of each parameter in influencing the membrane potential. 
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Membrane Potential Comparison 


e Healthy Brain: The membrane potential Fig. 12.8 for the healthy 
brain shows the typical action potential characteristics. 

e Parkinson's Brain: The membrane potential Fig. 12.8 for the 
Parkinson's brain, modified by changing parameters like dendrite 
diameter, membrane capacitance, and ion channel conductance, shows 
how the action potential is affected by Parkinson's disease. 


Key Observations 


e The differences in the membrane potential plots illustrate the impact 
of Parkinson's disease on neural activity. 


e The modified parameters in the Parkinson's model result in changes 
to the amplitude and shape of the action potentials, demonstrating the 
disease's effect on neuronal excitability. 


Suggestions for Parameters to Modify 


To explore the impact of different parameters on action potentials, you can 
modify the following inputs: 
1. Amplitude of Current (nA): 
e Higher amplitudes can increase the likelihood of triggering an 
action potential. 
e Lower amplitudes might not reach the threshold for an action 
potential. 
2. Duration of Current (ms): 
e Longer durations can sustain depolarization, potentially leading to 
multiple action potentials. 
e Shorter durations might only cause a brief depolarization without 
reaching the threshold. 
3. Membrane Capacitance (cm): 
e Increasing capacitance makes the membrane less responsive to 
voltage changes, slowing down action potential initiation. 
e Decreasing capacitance makes the membrane more responsive, 
speeding up action potential initiation. 
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4. Axial Resistance (Ra): 
e Higher resistance can reduce the speed of signal propagation along 
the neuron. 
e Lower resistance can increase the speed of signal propagation. 
5. Sodium Conductance (gbar_na): 
e Increasing sodium conductance can enhance the neuron's 
excitability, making it easier to trigger action potentials. 
e Decreasing sodium conductance can reduce excitability. 
6. Potassium Conductance (gbar_k): 
e Increasing potassium conductance can lead to quicker 
repolarization and a shorter action potential duration. 
e Decreasing potassium conductance can prolong repolarization and 
action potential duration. 
7. Leak Conductance (gbar_1): 
e Higher leak conductance can stabilize the resting membrane 
potential but reduce overall excitability. 
e Lower leak conductance can increase excitability but may make 
the neuron more prone to spontaneous activity. 
e Resting Membrane Potential (v_init): 
e A more negative resting potential (hyperpolarized) makes 
reaching the threshold for an action potential harder. 
e A less negative resting potential (depolarized) makes it easier to 
reach the threshold. 
What the Output Means 


Action Potential Amplitude: Indicates the strength of the neural 
signal. Changes in amplitude can reflect alterations in ion channel 
conductance or membrane properties. 

Action Potential Duration: Affects the timing and pattern of neural 
signaling. Prolonged action potentials can indicate issues with ion 
channel kinetics. 


513 


e Threshold: The minimum membrane potential required to trigger an 
action potential. Changes in the threshold can indicate alterations in 
membrane excitability. 

e Repolarization Phase: The phase during which the membrane 
potential returns to the resting state. Alterations in this phase can 
reflect changes in potassium conductance or other repolarizing 
mechanisms. 

By systematically modifying these parameters, you can gain insights into how 
different factors contribute to the overall behavior of neurons in both healthy 
and diseased states. This understanding can help in developing targeted 
therapies for neurological disorders like Parkinson's disease. 


rt 
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Chapter 12: Lab Example 2 


Overview 

In this lab example, we will combine some concepts you 
have learned from the textbook: neurostimulation (Chapter 
7), BCIs (Chapter 6), and disorders (Chapter 12). As you have learned, 
neurostimulation can modulate neural activity to alleviate symptoms of 


conditions such as Parkinson's disease, chronic pain, epilepsy, and depression. 
For instance, deep brain stimulation (DBS) is a widely used form of 
neurostimulation that targets brain regions to reduce tremors and improve 
motor function in Parkinson's patients. Similarly, spinal cord stimulation 
(SCS) is employed to manage chronic pain by altering pain signal transmission 
in the spinal cord. In this lab example, we will use the coil design from Chapter 
6: Lab Example to simulate neurostimulation between simulated Parkinson’s 
Disease and Depression. Then, we will do some statistical analysis (like the 
one in Chapter 5: Lab Example 2) to see how neurostimulation impacts these 
disorders differently. 


Requirements 
MATLAB (tested with R2021a and later) 


Steps 


The first thing you need to do is go to the GitHub repository for this book. You 
can find it using this link: 


https://github.com/bddupre92/Neurobook BME UND 


After you have opened the GitHub, you need to navigate to Chapter 12 Lab 
Example 2. In this lab example, you will find three scripts. 

e Rtmscoildepression.m 

e =Rtmscoilforjustparkinsons.m 

e = Ttestrtmscoil.m 
You will want to copy and paste each code into its separate script. 
We will run the RTMS Coil for Parkinson’s and then the RTMS Coil for 
Depression. The scripts follow the following structure. 
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Script Structure 


Define Constants and Parameters 

mu0: Permeability of free space. 

num_points: Number of points in each dimension for the grid. 

coil radius, coil center, frequencies, currents, phases: Parameters defining the 
neurostimulation coils. 

Observation Grid 

Define a 3D grid to simulate the brain model. 

Baseline Neural Activity Simulation: 

Simulate baseline neural activity with altered oscillatory patterns for 
depression. 

Magnetic Stimulation 

Calculate the magnetic field produced by the neurostimulation coils. 
Post-Stimulation Neural Activity Simulation: 

Simulate neural activity after magnetic stimulation. 

Time-Varying MEG Signal Calculation: 

Calculate the MEG signal over time. 

Visualization 


Plot the magnetic field slices before and after stimulation. 
Plot the MEG signal over time. 


Major Functions Used in the Script 
meshgrid 
e Usage: [x, y, z] = meshgrid(linspace(-0.1, 0.1, num_points), linspace(- 
0.1, 0.1, num_points), linspace(-0.1, 0.1, num_points)); 
e Purpose: Creates a 3D grid of points representing the observation 
space for the brain model. 
space 
e Usage: linspace(-0.1, 0.1, num_points) 
e Purpose: Generates linearly spaced vectors for defining the grid 
points in the x, y, and z dimensions. 
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sin 
e Usage: sin(2 * pi * f_beta * t(i)) 
e Purpose: Calculates the sine of an angle, which is used here to 
simulate oscillatory patterns in the MEG signals. 


subplot 
e Usage: subplot(2, 2, 1); 
e Purpose: Creates a subplot in a figure for plotting multiple graphs in 
a single window. 
imagesc 
e Usage: imagesc(linspace(-0.1, 0.1, num_points), linspace(-0.1, 0.1, 
num_points), meg_baseline_slice); 
e Purpose: Displays a scaled color image of the 2D matrix to visualize 
the magnetic field slices. 


Squeeze 
e Usage: meg_ baseline slice = squeeze(meg_baseline(:, :, slice_index)); 
e Purpose: Removes singleton dimensions from an array used here to extract 
a 2D slice from the 3D magnetic field data. 


e Usage: plot(t, meg _baseline_time(:, slice_index), 'DisplayName’, 'Baseline’); 
e Purpose: Plot data points in 2D space, which are used here to visualize the 
MEG signals over time. 


hold on/hold off 
e Usage: hold on; and hold off; 
e Purpose: Retains the current plot and certain settings so that subsequent 
graphing commands add to the existing graph. 


legend 

e ~=Usage: legend; 

e Purpose: Adds a legend to the plot to label different data series. 
title 

e Usage: title('Baseline Magnetic Field (z=0)'); 

e Purpose: Adds a title to the plot. 
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xlabel / ylabel 

e Usage: xlabel('x (m)'); and ylabel(‘y (m)’); 

e Purpose: Add labels to the plot on the x-axis and y-axis. 
colorbar 

e Usage: colorbar; 

e Purpose: Adds a color bar to the plot to indicate the color scale. 
clim 

e ~=Usage: clim([cmin cmax]); 


e Purpose: Set the plot's color limits to ensure consistent color scaling across 


different plots. 
When you run the functions, you will generate two figures. One for 


Parkinson’s Disease will look like Fig. 12.9, and one for depression will look 
like Fig. 12.10. 
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Figure 12.9: MEG Signal Over Time for Parkinson's Disease 
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Figure 12.10: MEG Signal Over Time for Depression 


Explanation of the Figures 


These figures demonstrate the simulation results of neurostimulation effects 
on MEG signals in a simplified brain model. Here are explanations of each 
plot of the statistics. 


Top Row: Magnetic Field Slices 


Left Plot: Baseline Magnetic Field (z=0) 
e This plot shows the baseline magnetic field slice at the middle plane 
of the brain model (z=0) before any stimulation is applied. 
e The color bar indicates the strength of the magnetic field in Tesla (T), 
with blue regions representing negative field values and yellow 
regions representing positive field values. 


Right Plot: Post-Stimulation Magnetic Field (z=0) 


e This plot shows the magnetic field slice at the middle plane of the 
brain model (z=0) after the neurostimulation has been applied. 

e The color bar again indicates the strength of the magnetic field in Tesla 
(T). 

e The difference between the baseline and post-stimulation plots 
demonstrates the effect of neurostimulation on the magnetic field. 
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Bottom Row: MEG Signal Over Time 
Plot: MEG Signal Over Time 


This plot shows the MEG signal at the brain model's central point 
(slice) over time (1 second). 
The x-axis represents time in seconds (s), and the y-axis repre- 
sents the magnetic field in Tesla (T). 
Two lines are plotted: 
= Blue Line (Baseline): Represents the MEG signal be- 
fore stimulation. 
= Orange Line (Post-Stimulation): Represents the 
MEG signal after stimulation. 
The differences in amplitude and phase between the baseline and 
post-stimulation signals indicate how neurostimulation affects 
neural oscillations and activity. 


Key Observations 


Magnetic Field Slices: The baseline and post-stimulation mag- 
netic field slices show how the applied stimulation alters the 
magnetic field distribution within the brain model. 

MEG Signal Over Time: The post-stimulation MEG signal ex- 
hibits slight changes in amplitude and phase compared to the 
baseline, illustrating the impact of neurostimulation on neural ac- 
tivity. These changes can be indicative of altered neural oscilla- 
tory patterns due to stimulation. 


Statistical Analysis 


Both figures look similar, and it is hard to understand if there are any 


differences. This is why it is important to understand statistical analysis 
techniques and how to use them in MATLAB. In this example, Please refer to 
Chapter 5: Lab Example 2 for a detailed explanation of a t-test. Go back to the 
final code in MATLAB called t-test rtms coil and click run. 
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Script Structure 


Perform t-test 


Use ttest2 to compare the difference in MEG signal changes between 
depression and Parkinson's disease. 


Display Results 


Print the p-value, t-statistic, degrees of freedom, and conclusion about the null 
hypothesis based on the t-test results. 


Major Functions Used in the Script 


ttest2 


e Usage: [~, p value diff, ~, stats diff] = ttest2(meg_post(:) - 
meg_baseline_alpha(:), meg_post_beta(:) - meg_baseline_beta(:)); 

e Purpose: Performs a two-sample t-test to compare the means of two 
independent samples. This script is used to compare the differences in 
MEG signals before and after stimulation between depression and 
Parkinson's disease conditions. 

e Outputs: 

o p_value_diff: The p-value of the t-test. 
o stats diff: A structure containing the t-statistic, degrees of 
freedom, and other relevant information. 


disp 
e Usage: disp(['P-value for t-test for the difference between Depression 
and Parkinson's: ', num2str(p_value_diff)]); 
e Purpose: Displays text or variables in the Command Window. It is 


used here to print the p-value, t-statistic, degrees of freedom, and 
conclusion about the null hypothesis. 


num2str 


e Usage: num2str(p_value_diff) 
e Purpose: Converts numbers to a string format. This is useful for 
concatenating numerical values with text for display purposes. 
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Once you have run this code, you will get some results in your Command 
Window. 


Explanation of the Results 


The statistical analysis performed using a two-sample t-test compares the 
differences in the effects of neurostimulation between depression and 
Parkinson's disease. The results of the t-test are as follows: 


P-value: 0.012806 


The p-value measures the probability that the observed differences between 
the two conditions could have occurred by random chance. A p-value less than 
0.05 is typically considered statistically significant. 

In this case, the p-value is 0.012806, less than 0.05. This indicates that the 
differences in the effects of neurostimulation between depression and 
Parkinson's disease are statistically significant. 


t-statistic: -2.4891 


The t-statistic measures the size of the difference relative to the variation in the 
sample data. A larger absolute value of the t-statistic indicates a more 
significant difference between the groups. 

Here, the t-statistic is -2.4891, which suggests a noticeable difference between 
the two conditions. 


Degrees of Freedom: 249998 


Degrees of freedom (df) refers to the number of independent values that can 
vary in the analysis. In the context of a t-test, it is used to determine the 
distribution of the test statistic. 

The degrees of freedom for this test are 249998, which is typical for a large 
dataset. 
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Results Conclusion 


We reject the null hypothesis based on the p-value being less than 0.05. The 
null hypothesis states that there is no significant difference in the effects of 
neurostimulation between depression and Parkinson's disease. 

The results indicate that there is a significant difference in how 
neurostimulation affects individuals with depression compared to those with 
Parkinson's disease. 


Summary 


In this lab example, we explored the effects of neurostimulation on neural 
activity for two neurological conditions: Parkinson's disease and depression. 
Using MATLAB scripts, we simulated the magnetic fields and MEG signals 
before and after neurostimulation for both conditions and then performed a 
statistical analysis to compare the differences in effects. 


The key findings from this lab include: 


Simulation Results 

The magnetic field distribution and MEG signals were visualized for baseline 
and post-stimulation conditions. 

Differences in amplitude and phase between baseline and post-stimulation 
MEG signals were observed, indicating the impact of neurostimulation on 
neural activity. 


Statistical Analysis 

A two-sample t-test was performed to statistically compare the effects of 
neurostimulation between depression and Parkinson's disease. 

The p-value obtained (0.012806) was less than the significance threshold of 
0.05, indicating a statistically significant difference in the effects of 
neurostimulation between the two conditions. 

The t-statistic (-2.4891) further confirmed the noticeable difference, and the 
degrees of freedom (249998) highlighted the robustness of the analysis due to 
the large dataset. 
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Interpretation 


The results indicate that neurostimulation affects individuals with depression 
differently compared to those with Parkinson's disease. This underscores the 
importance of tailoring neurostimulation treatments to specific neurological 
conditions to achieve the best therapeutic outcomes. 

This lab example combines concepts from neurostimulation, brain-computer 
interfaces (BCIs), and neurological disorders to comprehensively understand 
how neurostimulation can modulate neural activity differently across various 
conditions. The statistical analysis demonstrates the importance of using 
quantitative methods to validate the efficacy and specificity of 
neurostimulation treatments, highlighting the potential for personalized 
therapeutic approaches in clinical practice. 

Overall, this lab example practically applies theoretical knowledge, 
reinforcing the significance of interdisciplinary approaches in biomedical 
engineering and neuroscience. 


A 
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General Conclusion 


This book, "Introduction to Neuroengineering," has aimed to provide a 
comprehensive overview of the principles and applications within this 
interdisciplinary field. The convergence of neuroscience, engineering, and 
technology represents a frontier with potential for innovation in medical 
devices, neural interfaces, and therapeutic strategies. By offering this resource 
as an Open Educational Resource (OER), we hope to foster a collaborative and 
accessible learning environment for students, educators, and professionals. 
The journey through this book reflects the collective efforts and dedication of 
numerous individuals whose contributions have been invaluable. 
Neuroengineering, as explored throughout this book, stands at the intersection 
of multiple disciplines, drawing on advances in neural science, electrical 
engineering, computer science, and biomedical engineering. This approach 
allows for novel solutions to complex problems related to the human nervous 
system. From neural prosthetics that restore lost functions to brain-computer 
interfaces that open new avenues for communication and control, the 
innovations within neuroengineering are geared to revolutionize both 
healthcare and our understanding of the human brain. 

One of the key themes cultivated in this book is the importance of 
interdisciplinary collaboration. The field of neuroengineering thrives on the 
integration of diverse perspectives and expertise. By bringing together 
neuroscientists, engineers, clinicians, and computer scientists, we can tackle 
challenges that no single discipline could address alone. This collaborative 
spirit is essential for advancing research and translating discoveries into 
practical applications that can benefit society. 

As we conclude this journey through the foundational concepts and cutting- 
edge advancements in neuroengineering, it is important to reflect on our work's 
ethical considerations and societal implications. The technologies and 
techniques developed in this field have the potential to impact human lives 
profoundly. Therefore, we must approach our research and development 
efforts with a strong ethical framework, ensuring that the benefits of 
neuroengineering are accessible to all and the potential risks are carefully 
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managed. Ethical considerations, such as privacy, consent, and the long-term 
effects of neural interventions, must remain at the forefront of our discussions 
and decisions. 

Education and outreach also play a pivotal role in the future of 
neuroengineering. By making this book available as an OER, we aim to 
democratize and decolonize access to knowledge and inspire the next 
generation of neuroengineers. We hope students and educators from diverse 
backgrounds will use this resource to gain a solid foundation in the field and 
contribute their unique perspectives and ideas. The future of neuroengineering 
depends on the continuous influx of fresh, innovative minds eager to push the 
boundaries of what is possible. 

Furthermore, the practical applications of neuroengineering are great and 
varied. This book has explored several key areas, including neural prosthetics, 
brain-machine interfaces, neural imaging, and neuromodulation. Each area 
holds promise for significant advancements in medical treatment and quality 
of life. For example, neural prosthetics can potentially restore mobility and 
independence to individuals with limb loss, while brain-machine interfaces 
could provide new communication channels for those with severe motor 
impairments. Advances in neural imaging can lead to better diagnostic tools, 
and a deeper understanding of brain disorders, and neuromodulation 
techniques offer new avenues for treating conditions such as epilepsy and 
depression. 

The journey of creating this book has been a testament to the power of 
collaboration and the collective pursuit of knowledge. The support and 
contributions from various individuals and institutions have been instrumental 
in bringing this project to fruition. As we move forward, it is essential to 
continue fostering an environment of collaboration, innovation, and ethical 
responsibility. The challenges we face in neuroengineering are complex, but 
they are also opportunities to make a meaningful impact on human health and 
well-being. 
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In summary, "Introduction to Neuroengineering" is more than just a textbook; 
it is a stepping stone toward a future where the boundaries between biology 
and technology are seamlessly integrated for the betterment of humanity. We 
hope this resource will inspire, educate, and empower readers to contribute to 
the exciting and ever-evolving field of neuroengineering. The journey does not 
end here; it is only the beginning. As discoveries are made and technologies 
are developed, the principles and ideas discussed in this book will continue to 
evolve, driving the field forward and opening up new possibilities for 
understanding and enhancing the human experience. 
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